-- Logs begin at Thu 2022-07-21 05:49:20 UTC, end at Thu 2022-07-21 06:47:10 UTC. -- Jul 21 06:15:56.742961 dv-cinder-850318-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 06:15:56.750229 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 06:15:56.752217 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 06:15:56 2022] *** Jul 21 06:15:56.752217 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 06:15:56.752217 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 06:15:56.752217 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: nodename: dv-cinder-850318-3-iscsi Jul 21 06:15:56.752217 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: machine: x86_64 Jul 21 06:15:56.752217 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: clock source: unix Jul 21 06:15:56.752217 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: pcre jit disabled Jul 21 06:15:56.752217 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: detected number of CPU cores: 4 Jul 21 06:15:56.752719 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: current working directory: / Jul 21 06:15:56.752719 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: detected binary path: /usr/bin/uwsgi-core Jul 21 06:15:56.752719 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: your processes number limit is 47680 Jul 21 06:15:56.752719 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: your memory page size is 4096 bytes Jul 21 06:15:56.752719 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: detected max file descriptor number: 2048 Jul 21 06:15:56.752719 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: lock engine: pthread robust mutexes Jul 21 06:15:56.752719 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: thunder lock: enabled Jul 21 06:15:56.752719 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 06:15:56.752719 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 06:15:56.798203 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: Python main interpreter initialized at 0x55bceb36d9c0 Jul 21 06:15:56.798203 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: python threads support enabled Jul 21 06:15:56.798203 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: your server socket listen backlog is limited to 100 connections Jul 21 06:15:56.798203 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: your mercy for graceful operations on workers is 90 seconds Jul 21 06:15:56.798579 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 06:15:56.798655 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: *** Operational MODE: preforking *** Jul 21 06:15:56.798929 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: *** uWSGI is running in multiple interpreter mode *** Jul 21 06:15:56.798929 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: spawned uWSGI master process (pid: 80135) Jul 21 06:15:56.799128 dv-cinder-850318-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 06:15:56.801716 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: spawned uWSGI worker 1 (pid: 80141, cores: 1) Jul 21 06:15:56.801716 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: spawned uWSGI worker 2 (pid: 80142, cores: 1) Jul 21 06:15:56.801716 dv-cinder-850318-3-iscsi devstack@keystone.service[80135]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 06:16:02.275759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 06:16:02.275759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: from cryptography.utils import int_from_bytes Jul 21 06:16:02.276537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 06:16:02.276537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: from cryptography.utils import int_from_bytes Jul 21 06:16:02.276668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 06:16:02.276668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: from cryptography.utils import int_from_bytes Jul 21 06:16:02.276668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 06:16:02.276668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: from cryptography.utils import int_from_bytes Jul 21 06:16:02.325192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 06:16:02.325336 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 06:16:02.325458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 06:16:02.325595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] command line args: [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 06:16:02.325686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 06:16:02.325771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 06:16:02.325845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] command line args: [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 06:16:02.325916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ================================================================================ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 06:16:02.326139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 06:16:02.326229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] admin_token = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.326307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ================================================================================ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 06:16:02.326430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_dir = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.326581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] admin_token = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.326661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.326786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_source = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.326870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_dir = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.326948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] debug = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.327065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.327151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.327379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_source = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.327441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] debug = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.327507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.327665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.327771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.327914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.327996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.328121 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.328206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.328375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.328459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.328574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_config_append = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.328654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.328730 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.328870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.328957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_dir = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.329070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.329154 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_config_append = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.329270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.329355 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.329423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_dir = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.329535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.329608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.329757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.329884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.330040 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.330156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.330236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.330377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.330456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.330615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.330747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.330851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.330918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.330979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.331082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.331216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.331306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.331466 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.331562 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.331708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.331793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.331961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.332080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.332196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.332287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.332447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.332540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.333247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.333487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.333612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.333696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] publish_errors = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.333888 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.334049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.334175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.334295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.334470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.334603 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.334727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] publish_errors = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.334802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.334991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] standard_threads = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.335109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.335225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.335314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.335437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.335521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.335636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.335782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.335853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_journal = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.335923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.335992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_json = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.336134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_stderr = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.336218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] standard_threads = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.336301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_syslog = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.336378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.336553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.336639 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.336769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.336859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.336938 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_journal = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.337010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.337114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_json = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.337250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.337329 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_stderr = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.337462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_syslog = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.337540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.337699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 06:16:02.337776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.337962 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.338043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.338210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.338289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.338446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.338524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.338694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.338777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.338943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.339028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.339183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.339266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.339415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.339493 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.339661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.339741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.339896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.339974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.340143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.340220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.340390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.340470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.340626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.340714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.340875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.340953 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.341101 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.341181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.341355 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.341439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.341589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.341667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.341837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.341921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.342123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.342207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.342373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.342451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.342620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.342707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.342850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.342936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.343079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.343157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.343308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.343387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.343542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.343626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.343788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.343867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.344032 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.344117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.344260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.344339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.344490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.344581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.344727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.344806 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.344969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.345051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.345205 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.345291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.345438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.345522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.345672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.345750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.345904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.346021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.346165 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.346244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.346394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.346472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.346623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.346776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.346852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.346929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.347068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.347146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.347303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.347389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.347550 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.347635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.347779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.347865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.348010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.348095 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.348235 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.348314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.349045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.349182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.349304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.349423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.349531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.349644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.350506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.350593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.350668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.350739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.350806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.350871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.350930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.350988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.350988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.351134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.351194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.351257 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.351316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.351374 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.351431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.351496 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.351791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.351976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.352115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.352243 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.352321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.352458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.352458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.352639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.352639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.352834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.352834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.353014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.353014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.353203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.353203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.353394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.353470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.353542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.353658 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.353741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.353887 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.353979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.354155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.354237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.354392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.354485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.354628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.354751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.354864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.354943 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.355109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.355190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.355348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.355478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.355611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.355744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.355894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.356020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.356154 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.356282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.356282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.356525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.356663 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.356791 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.356919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.357052 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.357181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.357315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.357453 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.357587 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.357723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.357851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.358024 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.358150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.358292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.358676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.358814 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.358947 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.359074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.359200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.359398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.359531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.359653 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.359791 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.359919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.360046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.360181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.360317 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.360457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.360584 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.360708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.360843 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.360970 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.361095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.361231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.361358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.361485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.361628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.361756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.361884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.362042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.362179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.362307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.362435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.362563 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.362692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.362827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.362955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.363083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.external = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.363211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.363338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.363475 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.363609 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.363745 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.363871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.363998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.364122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.password = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.364258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.364412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.token = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.364548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.external = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.364820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.364949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.365086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.365225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.365366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.365495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.365630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.password = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.365759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.365882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.token = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.365882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.366186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.366315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.366442 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.366570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.366570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.366809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.366945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.367084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.367212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.367340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.367468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.367604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.367740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.367869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.367995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.368122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.368251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.368379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.368515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.368643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.368781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.368909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.369036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.369163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.369300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.369426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.369555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.369684 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.369810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.369949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.370086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.370211 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.370337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.370461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.370595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.370719 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.370858 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.370984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.371109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.371233 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.371358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.371483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.371615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.371740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.371865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.371989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.372114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.372248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.372383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.372507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.372631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.372755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.372879 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.373003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.373139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.373264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.373389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.373513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.373644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.373768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.373901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.374065 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.374191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.374315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.374440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.374564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.374765 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.374892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.375017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.375141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.375273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.375405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.375531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.375654 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.375789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.375914 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.376039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.376171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.376304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.376429 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.376554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.376678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.376803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.376934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.377059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.377184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.377309 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.377502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.377637 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.377771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.377897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.378062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.378188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.378314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.378444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.378582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.378711 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.378838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.378964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.379098 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.379226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.379371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.379505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.379707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.379707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.379951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.380079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.380214 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.380341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.380471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.380600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.380727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.380855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.380989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.381118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.381256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.381385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.381512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.381640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.381774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.381901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.382044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.382169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.382293 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.382417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.382552 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.382678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.382802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.382802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.383046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.383171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.383304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.383429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.383554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.383679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.383803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.383928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.384064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.384189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.384313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.384437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.384561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.384699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.384833 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.384959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.385084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.385208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.385333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.385457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.385609 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.385754 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.385878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.386044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.386174 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.386302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.386441 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.386580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.386707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.386835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.386963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.387092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.387226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.387354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.387479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.387605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.387732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.387861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.387997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.388125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.388261 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.388388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.388514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.388650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.388776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.388905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.389033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.389160 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.389286 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.389420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.389551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.389680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.389805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.389805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.390090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.390231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.390361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.390490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.390618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.390747 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.390875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.391013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.391142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.391271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.391399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.391528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.391656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.391792 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.391930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.392061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.392195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.392325 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.392452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.392594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.392721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.392850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.392978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.393110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.393234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.393367 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.393493 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.393628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.393753 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.393877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.394018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.394161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.394291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.394420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.394547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.394675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.394803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.394940 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.395067 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.395697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.395826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.395910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.396048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.396192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.396376 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.396512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.396639 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.396763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.396889 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.397023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.397148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.397273 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.397398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.397522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.397647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.397790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.397915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.398097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.398226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.398355 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.398483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.398622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.398749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.398876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.399004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.399132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.399258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.399394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.399522 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.399660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.399788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.399916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.400048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.400251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.400381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.400509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.400509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.400746 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.400874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.401008 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.401008 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.401249 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.401387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.401387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.401627 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.401764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.401892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.402034 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.402159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.402286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.402421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.402548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.402672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.402797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.402922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.403047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.403188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.403315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.403440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.403565 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.403690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.403816 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.403948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.404074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.404198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.404323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.404447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.404573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.404705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.404841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.404966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.405113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.405236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.405236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.405479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.405605 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.405729 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.405853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.406023 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.406151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.406289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.406418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.406555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.406682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.406808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.406936 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.407070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.407197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.407323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.407452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.407578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.407705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.407839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.407966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.408092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.408233 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.408360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.408495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.408622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.408749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.408876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.409003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.409129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.409264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.409391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.409526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.409653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.409781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.409909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.410077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.410203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.410327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.410452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.410577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.410701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.411435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.411622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.411770 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.411896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.412021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.412147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.412282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.412423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.412549 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.412675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.412800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.412926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.413060 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.413185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.413310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.413435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.413560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.413685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.413817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.413995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.414123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.414264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.414393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.414522 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.414661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.414796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.414934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.415065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.415193 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.415321 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.415449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.415577 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.415713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.415713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.415954 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.416101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.416237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.416366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.416492 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.416629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.416757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.416885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.417013 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.417141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.417269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.417404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.417532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.417661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.417789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.417928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418078 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 06:16:02.418971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 06:16:02.418971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 06:16:02.443161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.443317 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.443402 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.443477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.443597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.443685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.451928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.452021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.480860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.481042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.481138 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.481279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.623949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.624146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.673497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.673727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.674036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.674276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.739140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.739375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.739564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.739673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.739754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.739829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.739903 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.740025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.740108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.740237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.740321 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.740441 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.765765 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.765890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.766013 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.766190 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.766278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.766398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.766481 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.766617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.766701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.766832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.766914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.767051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.767134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.767263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.778822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.778955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.792171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.792405 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.792540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.792626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.792772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.792856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.792994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.793086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.793215 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.793298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.793432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.793515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.793645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.793857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 06:16:02.811326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.811536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 06:16:02.811852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.811939 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.812062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 06:16:02.812148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 06:16:02.812335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.812417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.812582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 06:16:02.812664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.812843 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.812926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.813099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.813180 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.813354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.813610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.824282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.824558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.824828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.824911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.825111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.825210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.825401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.825680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.833421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.834121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.836763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.837043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.837326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.837407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.837661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.837752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.837960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.838045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.838267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.838349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.838540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.838623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.838815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.838896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.839087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.839168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.839362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.839444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.839627 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.839707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.839899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.840165 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.868545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.869541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.870176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.870455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.871202 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.871486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.875751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.875993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.876272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.876765 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.877046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.877327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.884187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.884464 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.885269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.885555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.889079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.889363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.889644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.889924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.890231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.890316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.890518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.890600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.890801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.890882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.891068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.891153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.891349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.891625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.891898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.892166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.910362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.910649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.911651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.911935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.915571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.916588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.919212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.919505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.920249 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.920553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.924483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.925569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.928145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.929393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.931898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.932194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.932485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.932769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.933071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.933201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.933351 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.933501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.933796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.934117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.934401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.934680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.953106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.953477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.953746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.954039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.954626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.955004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.955290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.955560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.962509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.964173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.964265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.964620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.965880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.966181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.967612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.967898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.968180 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.968463 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.968736 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.969315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.969594 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.969876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.970182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.970485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.982287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.982568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.982848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.983124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.983407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.983687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.983930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:02.984014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.984212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.984505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.984792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.985076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.985369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:02.985647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.003708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.006300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.007569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.009575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.010885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.012469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.012908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.014135 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.014416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.014560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.014706 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.016203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.016486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.016776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.022225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.023837 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.024289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.025920 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.027075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.027368 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.027652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.027934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.029182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.029475 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.029763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.030076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.037195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.037477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.037752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.039411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.039700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.039983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.045721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.046020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.046302 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.046574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.046842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.047117 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.047393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.047659 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.047920 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 06:16:03.048006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.048193 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.048275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.048470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.048560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.048729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.048811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.048989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.049142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.049260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.049341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.049548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.049818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.050122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.053076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.053076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.053076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.053076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.053076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 06:16:03.089522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55bceb36d9c0 pid: 80142 (default app) Jul 21 06:16:03.091024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55bceb36d9c0 pid: 80141 (default app) Jul 21 06:16:03.092943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20db090-68e8-4814-a28c-a8c8c6eaef58 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:03.093238 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20db090-68e8-4814-a28c-a8c8c6eaef58 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:03.093521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20db090-68e8-4814-a28c-a8c8c6eaef58 None None] PATH_INFO: `/v3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:03.097768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Thu Jul 21 06:16:03 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:10.827087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077bde9-19e7-417d-b050-5399123781bb None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:10.827669 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077bde9-19e7-417d-b050-5399123781bb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:10.827669 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077bde9-19e7-417d-b050-5399123781bb None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:10.832239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:16:10.837459 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc79f1-a464-4485-8dcb-234484742d95 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:10.837837 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc79f1-a464-4485-8dcb-234484742d95 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:10.838051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc79f1-a464-4485-8dcb-234484742d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:10.906472 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: INFO dbcounter [None req-7ddc79f1-a464-4485-8dcb-234484742d95 None None] Registered counter for database keystone Jul 21 06:16:10.928171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG oslo_db.sqlalchemy.engines [None req-7ddc79f1-a464-4485-8dcb-234484742d95 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80142) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 06:16:10.934808 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writer thread running {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 06:16:11.112173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG passlib.handlers.bcrypt [None req-7ddc79f1-a464-4485-8dcb-234484742d95 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80142) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 06:16:11.112508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG passlib.handlers.bcrypt [None req-7ddc79f1-a464-4485-8dcb-234484742d95 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80142) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 06:16:11.169631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7ddc79f1-a464-4485-8dcb-234484742d95 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:11.171267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7ddc79f1-a464-4485-8dcb-234484742d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:11.293407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 21 06:16:11.462445 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077bde9-19e7-417d-b050-5399123781bb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:11.533180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: INFO dbcounter [None req-4077bde9-19e7-417d-b050-5399123781bb None None] Registered counter for database keystone Jul 21 06:16:11.551982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG oslo_db.sqlalchemy.engines [None req-4077bde9-19e7-417d-b050-5399123781bb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80141) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 06:16:11.558456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writer thread running {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 06:16:11.675056 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077bde9-19e7-417d-b050-5399123781bb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:11.679331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ff577-729e-4ecf-9c40-9e4d4e081836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:11.680004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ff577-729e-4ecf-9c40-9e4d4e081836 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:11.680294 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ff577-729e-4ecf-9c40-9e4d4e081836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:11.680831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ff577-729e-4ecf-9c40-9e4d4e081836 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:11.689459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ff577-729e-4ecf-9c40-9e4d4e081836 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:11.741053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7c0ff577-729e-4ecf-9c40-9e4d4e081836 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 06:16:11.741393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ff577-729e-4ecf-9c40-9e4d4e081836 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:11.751149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Jul 21 06:16:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:16:11.757164 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc79f1-a464-4485-8dcb-234484742d95 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:11.771403 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc79f1-a464-4485-8dcb-234484742d95 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:11.775741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26664843-c13e-4c6b-bf28-3d18ec8a50b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:11.776856 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26664843-c13e-4c6b-bf28-3d18ec8a50b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:11.777142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26664843-c13e-4c6b-bf28-3d18ec8a50b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:11.777420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26664843-c13e-4c6b-bf28-3d18ec8a50b3 admin admin] PATH_INFO: `/v3/projects/c098b85eefc14c3cac1c96ebb357b960` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:11.782074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26664843-c13e-4c6b-bf28-3d18ec8a50b3 admin admin] RBAC: Authorizing `identity:get_project(project_id=c098b85eefc14c3cac1c96ebb357b960)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:11.831638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: WARNING keystone.common.rbac_enforcer.enforcer [None req-26664843-c13e-4c6b-bf28-3d18ec8a50b3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 06:16:11.831972 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26664843-c13e-4c6b-bf28-3d18ec8a50b3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:11.836524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Thu Jul 21 06:16:11 2022] GET /identity/v3/projects/c098b85eefc14c3cac1c96ebb357b960 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:13.830150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcac5c-f021-4bf6-86a9-cbfab5a31db9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:13.830683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcac5c-f021-4bf6-86a9-cbfab5a31db9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:13.830778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcac5c-f021-4bf6-86a9-cbfab5a31db9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:13.834937 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:13.840201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de505fb8-a12d-4e1a-b0b6-beb6e1e36d3a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:13.840559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de505fb8-a12d-4e1a-b0b6-beb6e1e36d3a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:13.840774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de505fb8-a12d-4e1a-b0b6-beb6e1e36d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:13.859900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-de505fb8-a12d-4e1a-b0b6-beb6e1e36d3a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:13.861427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-de505fb8-a12d-4e1a-b0b6-beb6e1e36d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:13.869231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 06:16:13.892046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcac5c-f021-4bf6-86a9-cbfab5a31db9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:13.903936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcac5c-f021-4bf6-86a9-cbfab5a31db9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:13.908341 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b62be-198d-4775-82f0-b70d58c51fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:13.908975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b62be-198d-4775-82f0-b70d58c51fcf admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:13.909264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b62be-198d-4775-82f0-b70d58c51fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:13.909550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b62be-198d-4775-82f0-b70d58c51fcf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:13.917183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b62be-198d-4775-82f0-b70d58c51fcf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:13.918995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b62be-198d-4775-82f0-b70d58c51fcf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:13.927926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Jul 21 06:16:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:16:13.933409 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de505fb8-a12d-4e1a-b0b6-beb6e1e36d3a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:13.943124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de505fb8-a12d-4e1a-b0b6-beb6e1e36d3a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:13.947556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf3e0b-678d-435a-8767-86dfdd670ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:13.948411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf3e0b-678d-435a-8767-86dfdd670ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:13.948698 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf3e0b-678d-435a-8767-86dfdd670ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:13.948977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf3e0b-678d-435a-8767-86dfdd670ef5 admin admin] PATH_INFO: `/v3/users/a1033721c1da485882816b21d7d01a42` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:13.950405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf3e0b-678d-435a-8767-86dfdd670ef5 admin admin] RBAC: Authorizing `identity:get_user(user_id=a1033721c1da485882816b21d7d01a42)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:13.953088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf3e0b-678d-435a-8767-86dfdd670ef5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:13.954685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Thu Jul 21 06:16:13 2022] GET /identity/v3/users/a1033721c1da485882816b21d7d01a42 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:16.031869 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef60ab-5f7b-4023-9b8a-765b926ecb97 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.032408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef60ab-5f7b-4023-9b8a-765b926ecb97 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.032491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef60ab-5f7b-4023-9b8a-765b926ecb97 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.034007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:16.039010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5c27b-b6fb-4947-8db3-34d65f905e15 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.039448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5c27b-b6fb-4947-8db3-34d65f905e15 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.039580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5c27b-b6fb-4947-8db3-34d65f905e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.059911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cfe5c27b-b6fb-4947-8db3-34d65f905e15 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:16.061145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cfe5c27b-b6fb-4947-8db3-34d65f905e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:16.077018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c53d61-ed0f-48f4-92a5-f18c695831eb None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.077312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c53d61-ed0f-48f4-92a5-f18c695831eb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.078149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c53d61-ed0f-48f4-92a5-f18c695831eb None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.079524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:16:16.084530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374ff92-517a-4119-93cd-76c5e2f89717 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.084822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374ff92-517a-4119-93cd-76c5e2f89717 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.085109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374ff92-517a-4119-93cd-76c5e2f89717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.125361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG passlib.handlers.bcrypt [None req-3374ff92-517a-4119-93cd-76c5e2f89717 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80141) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 06:16:16.125774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG passlib.handlers.bcrypt [None req-3374ff92-517a-4119-93cd-76c5e2f89717 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80141) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 06:16:16.142593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:16.165876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5c27b-b6fb-4947-8db3-34d65f905e15 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:16.173014 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3374ff92-517a-4119-93cd-76c5e2f89717 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:16.174643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3374ff92-517a-4119-93cd-76c5e2f89717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:16.190354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:16.271213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5c27b-b6fb-4947-8db3-34d65f905e15 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:16.276995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:16:16.298743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374ff92-517a-4119-93cd-76c5e2f89717 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:16.305842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567fec7-bd83-49df-800a-0fdfa609c4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:16.306680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567fec7-bd83-49df-800a-0fdfa609c4ae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.306971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567fec7-bd83-49df-800a-0fdfa609c4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.307257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567fec7-bd83-49df-800a-0fdfa609c4ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.319015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567fec7-bd83-49df-800a-0fdfa609c4ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d567fec7-bd83-49df-800a-0fdfa609c4ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:16.330960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:16.331878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:16.332708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:16:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:16:16.338326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567fec7-bd83-49df-800a-0fdfa609c4ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:16.399558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374ff92-517a-4119-93cd-76c5e2f89717 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:16.401818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:16.420521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:16.438415 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d966a-48e4-442a-99c0-7dcc48af2587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:16.438977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567fec7-bd83-49df-800a-0fdfa609c4ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:16.439316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d966a-48e4-442a-99c0-7dcc48af2587 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.439607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d966a-48e4-442a-99c0-7dcc48af2587 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.439886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d966a-48e4-442a-99c0-7dcc48af2587 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.473267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e2a13-045b-4371-8f7f-4372459a2d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:16.473957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e2a13-045b-4371-8f7f-4372459a2d30 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.474251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e2a13-045b-4371-8f7f-4372459a2d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.474535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e2a13-045b-4371-8f7f-4372459a2d30 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.475294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e2a13-045b-4371-8f7f-4372459a2d30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:16.475606 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e2a13-045b-4371-8f7f-4372459a2d30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:16.477024 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e2a13-045b-4371-8f7f-4372459a2d30 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:16.489162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:16.494657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e2a13-045b-4371-8f7f-4372459a2d30 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:16.508154 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d966a-48e4-442a-99c0-7dcc48af2587 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:16.509699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d966a-48e4-442a-99c0-7dcc48af2587 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:16.593786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e2a13-045b-4371-8f7f-4372459a2d30 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:16.609594 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 06:16:16.615308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d966a-48e4-442a-99c0-7dcc48af2587 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:16.618225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:16.628847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace6d4e-301e-4dbc-bc78-255019498f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:16.629728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace6d4e-301e-4dbc-bc78-255019498f48 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.630039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace6d4e-301e-4dbc-bc78-255019498f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.630318 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace6d4e-301e-4dbc-bc78-255019498f48 None admin] PATH_INFO: `/v3/users/a1033721c1da485882816b21d7d01a42` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.631534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace6d4e-301e-4dbc-bc78-255019498f48 None admin] RBAC: Authorizing `identity:get_user(user_id=a1033721c1da485882816b21d7d01a42)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:16.632998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace6d4e-301e-4dbc-bc78-255019498f48 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:16.634830 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:16 2022] GET /identity/v3/users/a1033721c1da485882816b21d7d01a42 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:16.640127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace6d4e-301e-4dbc-bc78-255019498f48 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:16.654614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:16.713380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d966a-48e4-442a-99c0-7dcc48af2587 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:16.741339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace6d4e-301e-4dbc-bc78-255019498f48 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:16.747407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec05a8-3043-42e7-b9e9-ac58b882145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:16.748203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec05a8-3043-42e7-b9e9-ac58b882145d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.748488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec05a8-3043-42e7-b9e9-ac58b882145d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.748769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec05a8-3043-42e7-b9e9-ac58b882145d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.766841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec05a8-3043-42e7-b9e9-ac58b882145d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:16.768314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec05a8-3043-42e7-b9e9-ac58b882145d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:16.776884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dda40-fc1c-41ec-a4be-a5150c71231d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:16.777771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dda40-fc1c-41ec-a4be-a5150c71231d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.778084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dda40-fc1c-41ec-a4be-a5150c71231d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.778362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dda40-fc1c-41ec-a4be-a5150c71231d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.779552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150dda40-fc1c-41ec-a4be-a5150c71231d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:16.780991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150dda40-fc1c-41ec-a4be-a5150c71231d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:16.782780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:16.788246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dda40-fc1c-41ec-a4be-a5150c71231d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-38ec05a8-3043-42e7-b9e9-ac58b882145d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:16:16.788962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:16.789904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:16.790868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:16:16.790868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:16:16.790868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:16:16.790868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:16.790868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:16:16.796392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec05a8-3043-42e7-b9e9-ac58b882145d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:16.854131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:16.869362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:16.889786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dda40-fc1c-41ec-a4be-a5150c71231d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:16.897800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec05a8-3043-42e7-b9e9-ac58b882145d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:16.924698 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce72cd-7961-4a48-9bc2-7ed46f7bdc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:16.925350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce72cd-7961-4a48-9bc2-7ed46f7bdc1b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.925633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce72cd-7961-4a48-9bc2-7ed46f7bdc1b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.925908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce72cd-7961-4a48-9bc2-7ed46f7bdc1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.926774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce72cd-7961-4a48-9bc2-7ed46f7bdc1b None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.domain.id=default, user.id=a1033721c1da485882816b21d7d01a42) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:16.927337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce72cd-7961-4a48-9bc2-7ed46f7bdc1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:16.928779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce72cd-7961-4a48-9bc2-7ed46f7bdc1b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:16.933071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59053044-59e0-4555-a559-eabcf329abd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:16.933774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59053044-59e0-4555-a559-eabcf329abd8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:16.934178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59053044-59e0-4555-a559-eabcf329abd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:16.934469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59053044-59e0-4555-a559-eabcf329abd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:16.935208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59053044-59e0-4555-a559-eabcf329abd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:16.935525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59053044-59e0-4555-a559-eabcf329abd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:16.935862 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Jul 21 06:16:16 2022] GET /identity/v3/role_assignments?user.id=a1033721c1da485882816b21d7d01a42&scope.domain.id=default&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:16.937077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59053044-59e0-4555-a559-eabcf329abd8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:16.956566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:19.068632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b82bd-9317-4f8e-a5fd-c898c0814631 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:19.069099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b82bd-9317-4f8e-a5fd-c898c0814631 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:19.069270 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b82bd-9317-4f8e-a5fd-c898c0814631 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:19.070774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:19.076425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647e561-9ee4-4022-85bc-6df3c30f794c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:19.077228 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647e561-9ee4-4022-85bc-6df3c30f794c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:19.077703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647e561-9ee4-4022-85bc-6df3c30f794c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:19.103105 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e647e561-9ee4-4022-85bc-6df3c30f794c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:19.105177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e647e561-9ee4-4022-85bc-6df3c30f794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:19.192335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 11/23] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:19.231413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b82bd-9317-4f8e-a5fd-c898c0814631 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:19.399288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b82bd-9317-4f8e-a5fd-c898c0814631 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:19.415916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600e14a-8642-4d15-99c4-50384b012cf5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:19.416598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600e14a-8642-4d15-99c4-50384b012cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:19.417096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600e14a-8642-4d15-99c4-50384b012cf5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:19.418703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 12/24] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:19.423657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e00cf-d71b-40ea-85b1-2aa657572a66 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:19.424340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e00cf-d71b-40ea-85b1-2aa657572a66 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:19.424847 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e00cf-d71b-40ea-85b1-2aa657572a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:19.456698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-690e00cf-d71b-40ea-85b1-2aa657572a66 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:19.458244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-690e00cf-d71b-40ea-85b1-2aa657572a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:19.482655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cc76a-4a46-4844-818c-a7d3c3efbde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:19.483480 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cc76a-4a46-4844-818c-a7d3c3efbde3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:19.483766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cc76a-4a46-4844-818c-a7d3c3efbde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:19.484047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cc76a-4a46-4844-818c-a7d3c3efbde3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:19.511252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cc76a-4a46-4844-818c-a7d3c3efbde3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2a0cc76a-4a46-4844-818c-a7d3c3efbde3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:19.518934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:19.519812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:19.520568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 13/25] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:16:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:19.527074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cc76a-4a46-4844-818c-a7d3c3efbde3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:19.536933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:19.556121 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:19.611248 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 13/26] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:19.685499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e00cf-d71b-40ea-85b1-2aa657572a66 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:19.737901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cc76a-4a46-4844-818c-a7d3c3efbde3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:19.799866 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a902e-f21e-4875-a090-c26917b75dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:19.800806 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a902e-f21e-4875-a090-c26917b75dcf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:19.801092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a902e-f21e-4875-a090-c26917b75dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:19.801374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a902e-f21e-4875-a090-c26917b75dcf None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:19.802163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a902e-f21e-4875-a090-c26917b75dcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:19.802478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a902e-f21e-4875-a090-c26917b75dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:19.803890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a902e-f21e-4875-a090-c26917b75dcf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:19.817583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 14/27] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:19.820041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d99e9c-bcb6-4d75-bfde-a0ac52c5b698 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:19.820545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d99e9c-bcb6-4d75-bfde-a0ac52c5b698 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:19.821048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d99e9c-bcb6-4d75-bfde-a0ac52c5b698 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:19.822772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 15/28] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:19.824324 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d99e9c-bcb6-4d75-bfde-a0ac52c5b698 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:19.871423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e00cf-d71b-40ea-85b1-2aa657572a66 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:19.946521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd163582-6b79-4ee3-91e2-45be60731096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:19.947329 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd163582-6b79-4ee3-91e2-45be60731096 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:19.947730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd163582-6b79-4ee3-91e2-45be60731096 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:19.949166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd163582-6b79-4ee3-91e2-45be60731096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:19.961106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd163582-6b79-4ee3-91e2-45be60731096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:19.964186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:19.964405 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd163582-6b79-4ee3-91e2-45be60731096 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-bd163582-6b79-4ee3-91e2-45be60731096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:16:19.986940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:19.987986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:19.988994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:16:19.988994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:16:19.988994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:16:19.988994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:19.991166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 14/29] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:19.993016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d3d60-e303-4b96-b6c5-8bceccd977d7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:19.993408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d3d60-e303-4b96-b6c5-8bceccd977d7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:19.993821 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d3d60-e303-4b96-b6c5-8bceccd977d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:20.012899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5c4d3d60-e303-4b96-b6c5-8bceccd977d7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:20.014341 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5c4d3d60-e303-4b96-b6c5-8bceccd977d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:20.045558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d99e9c-bcb6-4d75-bfde-a0ac52c5b698 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:20.059586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:20.085345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03df914-d9d5-42f7-9c3e-ba339aa8b267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:20.086384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03df914-d9d5-42f7-9c3e-ba339aa8b267 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:20.086675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03df914-d9d5-42f7-9c3e-ba339aa8b267 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:20.086954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03df914-d9d5-42f7-9c3e-ba339aa8b267 None admin] PATH_INFO: `/v3/users/a1033721c1da485882816b21d7d01a42` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:20.088986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03df914-d9d5-42f7-9c3e-ba339aa8b267 None admin] RBAC: Authorizing `identity:get_user(user_id=a1033721c1da485882816b21d7d01a42)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:20.090625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03df914-d9d5-42f7-9c3e-ba339aa8b267 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:20.092335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 16/30] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:19 2022] GET /identity/v3/users/a1033721c1da485882816b21d7d01a42 => generated 247 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:20.093157 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03df914-d9d5-42f7-9c3e-ba339aa8b267 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:20.148524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 15/31] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:20.149396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d3d60-e303-4b96-b6c5-8bceccd977d7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:20.205479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03df914-d9d5-42f7-9c3e-ba339aa8b267 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:20.243753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae975b7a-9453-4803-a02e-7ff72bcbf24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:20.244477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae975b7a-9453-4803-a02e-7ff72bcbf24f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:20.244816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae975b7a-9453-4803-a02e-7ff72bcbf24f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:20.245217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae975b7a-9453-4803-a02e-7ff72bcbf24f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:20.245982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae975b7a-9453-4803-a02e-7ff72bcbf24f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:20.246306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae975b7a-9453-4803-a02e-7ff72bcbf24f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:20.247718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae975b7a-9453-4803-a02e-7ff72bcbf24f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:20.249538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:20.269068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 17/32] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:20.269969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae975b7a-9453-4803-a02e-7ff72bcbf24f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:20.299924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d3d60-e303-4b96-b6c5-8bceccd977d7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:20.340908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67730174-caf3-45f1-be5b-5b4894124f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:20.341721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67730174-caf3-45f1-be5b-5b4894124f29 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:20.342036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67730174-caf3-45f1-be5b-5b4894124f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:20.342323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67730174-caf3-45f1-be5b-5b4894124f29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:20.343766 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67730174-caf3-45f1-be5b-5b4894124f29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:20.345436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67730174-caf3-45f1-be5b-5b4894124f29 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:20.347094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 16/33] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:20.353379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1c8c-ebfd-4e2a-bc72-c2b9ba77ef0d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:20.353379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1c8c-ebfd-4e2a-bc72-c2b9ba77ef0d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:20.354150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1c8c-ebfd-4e2a-bc72-c2b9ba77ef0d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:20.355763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 17/34] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:20.356608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb1c8c-ebfd-4e2a-bc72-c2b9ba77ef0d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:20.385563 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae975b7a-9453-4803-a02e-7ff72bcbf24f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:20.386725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:20.426690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb23e49-786a-4014-97da-4bbf0f89598e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:20.427676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb23e49-786a-4014-97da-4bbf0f89598e None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:20.428166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb23e49-786a-4014-97da-4bbf0f89598e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:20.428667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb23e49-786a-4014-97da-4bbf0f89598e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:20.429755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb23e49-786a-4014-97da-4bbf0f89598e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:20.431540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb23e49-786a-4014-97da-4bbf0f89598e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:20.471033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb1c8c-ebfd-4e2a-bc72-c2b9ba77ef0d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:20.513569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dba2b-b4f4-428b-be35-e9c16e7aecd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:20.514581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dba2b-b4f4-428b-be35-e9c16e7aecd8 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:20.515013 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dba2b-b4f4-428b-be35-e9c16e7aecd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:20.515514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dba2b-b4f4-428b-be35-e9c16e7aecd8 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:20.516416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820dba2b-b4f4-428b-be35-e9c16e7aecd8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:20.518114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820dba2b-b4f4-428b-be35-e9c16e7aecd8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:20.574428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 18/35] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jul 21 06:16:20 2022] POST /identity/v3/roles => generated 219 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:20.575570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb23e49-786a-4014-97da-4bbf0f89598e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:20.683130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 18/36] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:16:20 2022] POST /identity/v3/projects => generated 301 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:20.685657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291308e9-ab56-40e9-9ea3-ad8999887333 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:20.686214 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291308e9-ab56-40e9-9ea3-ad8999887333 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:20.686654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291308e9-ab56-40e9-9ea3-ad8999887333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:20.709153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-291308e9-ab56-40e9-9ea3-ad8999887333 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:20.711212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-291308e9-ab56-40e9-9ea3-ad8999887333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:20.737282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:20.846420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 19/37] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:20.862398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb23e49-786a-4014-97da-4bbf0f89598e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:20.871184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291308e9-ab56-40e9-9ea3-ad8999887333 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:20.886766 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:20.925263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7b549-524d-453d-badb-50d761db5ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:20.926316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7b549-524d-453d-badb-50d761db5ff7 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:20.926602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7b549-524d-453d-badb-50d761db5ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:20.926886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7b549-524d-453d-badb-50d761db5ff7 None admin] PATH_INFO: `/v3/domains/default/users/a1033721c1da485882816b21d7d01a42/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:20.929231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7b549-524d-453d-badb-50d761db5ff7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a1033721c1da485882816b21d7d01a42, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:20.931649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7b549-524d-453d-badb-50d761db5ff7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:20.950899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 19/38] 10.222.0.42 () {58 vars in 1361 bytes} [Thu Jul 21 06:16:20 2022] PUT /identity/v3/domains/default/users/a1033721c1da485882816b21d7d01a42/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:16:20.952079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c65d6-3323-4d84-8073-fe818cb968ef None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:20.952375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c65d6-3323-4d84-8073-fe818cb968ef None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:20.952660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c65d6-3323-4d84-8073-fe818cb968ef None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:20.954116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 20/39] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:20.963508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af4454-709d-4df4-a1c7-fed4fa806261 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:20.963832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af4454-709d-4df4-a1c7-fed4fa806261 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:20.964085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af4454-709d-4df4-a1c7-fed4fa806261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:21.000175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a2af4454-709d-4df4-a1c7-fed4fa806261 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:21.002873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a2af4454-709d-4df4-a1c7-fed4fa806261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:21.016394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291308e9-ab56-40e9-9ea3-ad8999887333 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:21.065703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37a128-b832-4697-b748-f6579667efa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:21.066500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37a128-b832-4697-b748-f6579667efa9 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:21.066793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37a128-b832-4697-b748-f6579667efa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:21.067081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37a128-b832-4697-b748-f6579667efa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:21.068003 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37a128-b832-4697-b748-f6579667efa9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:21.072896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37a128-b832-4697-b748-f6579667efa9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:21.097395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 20/40] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jul 21 06:16:20 2022] POST /identity/v3/roles => generated 225 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:16:21.098668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e20f1-7b38-4477-aee3-33a889a87010 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:21.099009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e20f1-7b38-4477-aee3-33a889a87010 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:21.099295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e20f1-7b38-4477-aee3-33a889a87010 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:21.102980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 21/41] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:21.107682 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb86c35-ce74-49b6-8520-b63128c702ae None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:21.107983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb86c35-ce74-49b6-8520-b63128c702ae None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:21.108256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb86c35-ce74-49b6-8520-b63128c702ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:21.131960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 21/42] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:21.139031 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8eb86c35-ce74-49b6-8520-b63128c702ae None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:21.139347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8eb86c35-ce74-49b6-8520-b63128c702ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:21.157442 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af4454-709d-4df4-a1c7-fed4fa806261 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:21.220803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:21.224561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:21.229797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 22/43] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:21.258353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb86c35-ce74-49b6-8520-b63128c702ae None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:21.297217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af4454-709d-4df4-a1c7-fed4fa806261 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:21.387099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb86c35-ce74-49b6-8520-b63128c702ae None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:21.424953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dcde1-78c7-4d27-8694-05f8803b2706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:21.425637 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dcde1-78c7-4d27-8694-05f8803b2706 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:21.425931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dcde1-78c7-4d27-8694-05f8803b2706 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:21.426244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dcde1-78c7-4d27-8694-05f8803b2706 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:21.427009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2dcde1-78c7-4d27-8694-05f8803b2706 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:21.428410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2dcde1-78c7-4d27-8694-05f8803b2706 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:21.443056 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 23/44] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jul 21 06:16:21 2022] POST /identity/v3/roles => generated 223 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:16:21.448233 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de29064-422d-40c3-ad90-1f9931c52606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:21.449066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de29064-422d-40c3-ad90-1f9931c52606 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:21.449352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de29064-422d-40c3-ad90-1f9931c52606 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:21.449632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de29064-422d-40c3-ad90-1f9931c52606 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:21.451323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de29064-422d-40c3-ad90-1f9931c52606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:21.452841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de29064-422d-40c3-ad90-1f9931c52606 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:21.454828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 22/45] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:21.460942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dcde1-78c7-4d27-8694-05f8803b2706 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:21.538883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:21.589993 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dcde1-78c7-4d27-8694-05f8803b2706 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:21.628229 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2480a0-49b3-461c-a761-dbf511c8906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:21.628902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2480a0-49b3-461c-a761-dbf511c8906b None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:21.629186 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2480a0-49b3-461c-a761-dbf511c8906b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:21.629462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2480a0-49b3-461c-a761-dbf511c8906b None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:21.630198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2480a0-49b3-461c-a761-dbf511c8906b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:21.631644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2480a0-49b3-461c-a761-dbf511c8906b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:21.763075 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:16:21 2022] POST /identity/v3/projects => generated 312 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:23.075823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9d8db-f6b3-4998-8413-cd624c1b9d9b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:23.076240 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9d8db-f6b3-4998-8413-cd624c1b9d9b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:23.076390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9d8db-f6b3-4998-8413-cd624c1b9d9b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:23.077775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 23/47] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:23.083049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f49b91-d651-4094-86dd-2a2ee539cb0b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:23.083437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f49b91-d651-4094-86dd-2a2ee539cb0b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:23.083616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f49b91-d651-4094-86dd-2a2ee539cb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:23.100079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-34f49b91-d651-4094-86dd-2a2ee539cb0b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:23.101255 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-34f49b91-d651-4094-86dd-2a2ee539cb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:23.167057 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 25/48] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:23.190774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9d8db-f6b3-4998-8413-cd624c1b9d9b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:23.291719 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9d8db-f6b3-4998-8413-cd624c1b9d9b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:23.305406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:23.327521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497937f4-085e-4942-9d15-369c3ff27b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:23.328322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497937f4-085e-4942-9d15-369c3ff27b14 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:23.328608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497937f4-085e-4942-9d15-369c3ff27b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:23.328889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497937f4-085e-4942-9d15-369c3ff27b14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:23.339845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497937f4-085e-4942-9d15-369c3ff27b14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-497937f4-085e-4942-9d15-369c3ff27b14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:23.347844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:23.348709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:23.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 24/49] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:16:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:23.355720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f49b91-d651-4094-86dd-2a2ee539cb0b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:23.397209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:23.458429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f49b91-d651-4094-86dd-2a2ee539cb0b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:23.492844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89949791-08c3-4e54-9d34-3af4c06fe2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:23.493523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89949791-08c3-4e54-9d34-3af4c06fe2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:23.493814 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89949791-08c3-4e54-9d34-3af4c06fe2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:23.494127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89949791-08c3-4e54-9d34-3af4c06fe2d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:23.494897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89949791-08c3-4e54-9d34-3af4c06fe2d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:23.495210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89949791-08c3-4e54-9d34-3af4c06fe2d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:23.496611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89949791-08c3-4e54-9d34-3af4c06fe2d1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:23.510437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 26/50] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:23.515905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497937f4-085e-4942-9d15-369c3ff27b14 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:23.616022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497937f4-085e-4942-9d15-369c3ff27b14 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:23.649617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c986cff-2bc0-4f56-a200-4ad431b0f8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:23.650509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c986cff-2bc0-4f56-a200-4ad431b0f8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:23.650794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c986cff-2bc0-4f56-a200-4ad431b0f8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:23.651068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c986cff-2bc0-4f56-a200-4ad431b0f8d3 None admin] PATH_INFO: `/v3/users/a1033721c1da485882816b21d7d01a42` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:23.652249 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c986cff-2bc0-4f56-a200-4ad431b0f8d3 None admin] RBAC: Authorizing `identity:get_user(user_id=a1033721c1da485882816b21d7d01a42)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:23.653701 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c986cff-2bc0-4f56-a200-4ad431b0f8d3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:23.655365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 25/51] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:23 2022] GET /identity/v3/users/a1033721c1da485882816b21d7d01a42 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:23.660710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89949791-08c3-4e54-9d34-3af4c06fe2d1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:23.758228 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89949791-08c3-4e54-9d34-3af4c06fe2d1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:23.759737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:23.797664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b0633-9a89-418a-891f-62cd8161db96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:23.798510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b0633-9a89-418a-891f-62cd8161db96 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:23.798798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b0633-9a89-418a-891f-62cd8161db96 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:23.799078 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b0633-9a89-418a-891f-62cd8161db96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:23.800249 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b0633-9a89-418a-891f-62cd8161db96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:23.801707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b0633-9a89-418a-891f-62cd8161db96 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:23.803509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 27/52] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:23.809300 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c986cff-2bc0-4f56-a200-4ad431b0f8d3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:23.824633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f048eb9-8543-4eee-b400-993eb7cf673d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:23.824930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f048eb9-8543-4eee-b400-993eb7cf673d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:23.825212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f048eb9-8543-4eee-b400-993eb7cf673d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:23.826583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 28/53] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:16:23.831256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cff98c-5837-4e17-9f97-043083685b9a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:23.831544 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cff98c-5837-4e17-9f97-043083685b9a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:23.831824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cff98c-5837-4e17-9f97-043083685b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:23.849263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a3cff98c-5837-4e17-9f97-043083685b9a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:23.850496 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a3cff98c-5837-4e17-9f97-043083685b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:23.855304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:23.912451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c986cff-2bc0-4f56-a200-4ad431b0f8d3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:23.917355 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 29/54] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:16:23.941121 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cff98c-5837-4e17-9f97-043083685b9a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:23.946415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15504a84-4ee2-4396-9270-4821973306a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:23.947081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15504a84-4ee2-4396-9270-4821973306a8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:23.947364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15504a84-4ee2-4396-9270-4821973306a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:23.947643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15504a84-4ee2-4396-9270-4821973306a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:23.948485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15504a84-4ee2-4396-9270-4821973306a8 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.domain.id=default, user.id=a1033721c1da485882816b21d7d01a42) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:23.948802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15504a84-4ee2-4396-9270-4821973306a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:23.950296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15504a84-4ee2-4396-9270-4821973306a8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:23.956094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 26/55] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Jul 21 06:16:23 2022] GET /identity/v3/role_assignments?user.id=a1033721c1da485882816b21d7d01a42&scope.domain.id=default&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:24.045644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:24.049570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cff98c-5837-4e17-9f97-043083685b9a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:24.087653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a9f76-7c53-4345-9921-501e71888fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:24.088474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a9f76-7c53-4345-9921-501e71888fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:24.088767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a9f76-7c53-4345-9921-501e71888fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:24.089049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a9f76-7c53-4345-9921-501e71888fb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:24.090320 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a9f76-7c53-4345-9921-501e71888fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:24.091800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a9f76-7c53-4345-9921-501e71888fb0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:24.093498 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 30/56] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:24.099123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15504a84-4ee2-4396-9270-4821973306a8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:24.200713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15504a84-4ee2-4396-9270-4821973306a8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:24.202906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:24.233734 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9ee61-0ab1-4719-98b7-a0845dd106b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:24.234434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9ee61-0ab1-4719-98b7-a0845dd106b5 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:24.234725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9ee61-0ab1-4719-98b7-a0845dd106b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:24.235008 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9ee61-0ab1-4719-98b7-a0845dd106b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:24.235757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9ee61-0ab1-4719-98b7-a0845dd106b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:24.237214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9ee61-0ab1-4719-98b7-a0845dd106b5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:24.263238 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 27/57] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Jul 21 06:16:24 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:16:26.292397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5b441-5ae1-4258-a2c7-1fa06bd766a5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:26.292898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5b441-5ae1-4258-a2c7-1fa06bd766a5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:26.293003 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5b441-5ae1-4258-a2c7-1fa06bd766a5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:26.294423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 31/58] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:26.299765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec00e20-dea7-407a-bca5-8e0ca49be6d2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:26.300161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec00e20-dea7-407a-bca5-8e0ca49be6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:26.300339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec00e20-dea7-407a-bca5-8e0ca49be6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:26.317524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dec00e20-dea7-407a-bca5-8e0ca49be6d2 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:26.319243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dec00e20-dea7-407a-bca5-8e0ca49be6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:26.383821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 28/59] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:26.409584 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5b441-5ae1-4258-a2c7-1fa06bd766a5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:26.510768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5b441-5ae1-4258-a2c7-1fa06bd766a5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:26.543666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f123374-68f3-4797-85e6-22130c94064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:26.544453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f123374-68f3-4797-85e6-22130c94064a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:26.544738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f123374-68f3-4797-85e6-22130c94064a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:26.545021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f123374-68f3-4797-85e6-22130c94064a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:26.546458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f123374-68f3-4797-85e6-22130c94064a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:26.547888 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f123374-68f3-4797-85e6-22130c94064a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:26.549471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 32/60] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:26.555500 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec00e20-dea7-407a-bca5-8e0ca49be6d2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:26.637168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:26.652769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec00e20-dea7-407a-bca5-8e0ca49be6d2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:26.685737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c5bac-13d0-4d36-a24c-b95ea5cb9118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:26.686445 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c5bac-13d0-4d36-a24c-b95ea5cb9118 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:26.686718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c5bac-13d0-4d36-a24c-b95ea5cb9118 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:26.687000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c5bac-13d0-4d36-a24c-b95ea5cb9118 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:26.687710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c5bac-13d0-4d36-a24c-b95ea5cb9118 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:26.689169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c5bac-13d0-4d36-a24c-b95ea5cb9118 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:26.714307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 29/61] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:16:26 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:29.976968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98860d3d-a562-4190-9468-75dd11840715 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:29.976968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98860d3d-a562-4190-9468-75dd11840715 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:29.976968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98860d3d-a562-4190-9468-75dd11840715 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:29.977918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 33/62] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:29.984472 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8aad0-e07b-4025-8104-a8372fdaf076 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:29.985427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8aad0-e07b-4025-8104-a8372fdaf076 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:29.985913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8aad0-e07b-4025-8104-a8372fdaf076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.018352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-71e8aad0-e07b-4025-8104-a8372fdaf076 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:30.020009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-71e8aad0-e07b-4025-8104-a8372fdaf076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:30.020925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda456b-0a96-49fe-91f5-7e911f29bb05 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.021413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda456b-0a96-49fe-91f5-7e911f29bb05 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.021880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda456b-0a96-49fe-91f5-7e911f29bb05 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.023471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 34/63] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:30.031344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a99a-6f30-4f47-9668-70c68cdd4a7b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.031891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a99a-6f30-4f47-9668-70c68cdd4a7b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.032371 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a99a-6f30-4f47-9668-70c68cdd4a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.069227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-24d1a99a-6f30-4f47-9668-70c68cdd4a7b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:30.070800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-24d1a99a-6f30-4f47-9668-70c68cdd4a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:30.114374 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:30.140559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 30/64] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:30.164203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8aad0-e07b-4025-8104-a8372fdaf076 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:30.195834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 35/65] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:30.246584 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a99a-6f30-4f47-9668-70c68cdd4a7b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:30.262366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:30.307134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8aad0-e07b-4025-8104-a8372fdaf076 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:30.343719 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8b898-1a7b-4771-baae-be284bb72f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:30.344529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8b898-1a7b-4771-baae-be284bb72f88 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.344820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8b898-1a7b-4771-baae-be284bb72f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.345102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8b898-1a7b-4771-baae-be284bb72f88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.356800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8b898-1a7b-4771-baae-be284bb72f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-92e8b898-1a7b-4771-baae-be284bb72f88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:30.365433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:30.366358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:30.367128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 31/66] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:16:30.367775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a99a-6f30-4f47-9668-70c68cdd4a7b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:30.368338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9a65e-7d07-481d-8067-ef3e3db5b256 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.368338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9a65e-7d07-481d-8067-ef3e3db5b256 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.368509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9a65e-7d07-481d-8067-ef3e3db5b256 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.369844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 32/67] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:30.373964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f110-cb9d-478a-ac88-ed5b7bedd056 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.373964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f110-cb9d-478a-ac88-ed5b7bedd056 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.373964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f110-cb9d-478a-ac88-ed5b7bedd056 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.373964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 33/68] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:30.376230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354f110-cb9d-478a-ac88-ed5b7bedd056 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:30.411602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe3e74-8549-4947-8f92-d2e3045c70ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:30.412416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe3e74-8549-4947-8f92-d2e3045c70ce None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.412708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe3e74-8549-4947-8f92-d2e3045c70ce None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.412994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe3e74-8549-4947-8f92-d2e3045c70ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.424614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe3e74-8549-4947-8f92-d2e3045c70ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-78fe3e74-8549-4947-8f92-d2e3045c70ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:30.434149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:30.434999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:30.435767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 36/69] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:16:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:30.436625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bf08d-212d-491f-a9ee-66ee55bf30ca None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.436847 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bf08d-212d-491f-a9ee-66ee55bf30ca None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.437130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bf08d-212d-491f-a9ee-66ee55bf30ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.453948 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e95bf08d-212d-491f-a9ee-66ee55bf30ca None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:30.455152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e95bf08d-212d-491f-a9ee-66ee55bf30ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:30.475790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:30.483465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354f110-cb9d-478a-ac88-ed5b7bedd056 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:30.513198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:30.518498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3be6b-b694-4789-b158-e12537400317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:30.519311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3be6b-b694-4789-b158-e12537400317 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.519610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3be6b-b694-4789-b158-e12537400317 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.519886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3be6b-b694-4789-b158-e12537400317 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.520644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3be6b-b694-4789-b158-e12537400317 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:30.520958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3be6b-b694-4789-b158-e12537400317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:30.522383 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3be6b-b694-4789-b158-e12537400317 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:30.523966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 37/70] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:30.530487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c6c7-6b3b-49f3-bb92-da92830ac30d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.530788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c6c7-6b3b-49f3-bb92-da92830ac30d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.538331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c6c7-6b3b-49f3-bb92-da92830ac30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.538569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 34/71] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:30.538569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397dad9-9d2c-420c-95e8-bfcf7d05e2dd None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.538569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397dad9-9d2c-420c-95e8-bfcf7d05e2dd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.538569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397dad9-9d2c-420c-95e8-bfcf7d05e2dd None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.538569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 35/72] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:30.538569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397dad9-9d2c-420c-95e8-bfcf7d05e2dd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:30.549009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fb51c6c7-6b3b-49f3-bb92-da92830ac30d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:30.550774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fb51c6c7-6b3b-49f3-bb92-da92830ac30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:30.617854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 38/73] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:30.618713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51c6c7-6b3b-49f3-bb92-da92830ac30d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:30.644734 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397dad9-9d2c-420c-95e8-bfcf7d05e2dd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:30.679652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd78e28-a7f1-4f8f-8187-a038889cf94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:30.680327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd78e28-a7f1-4f8f-8187-a038889cf94f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.680616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd78e28-a7f1-4f8f-8187-a038889cf94f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.680905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd78e28-a7f1-4f8f-8187-a038889cf94f None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.681663 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd78e28-a7f1-4f8f-8187-a038889cf94f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:30.682006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd78e28-a7f1-4f8f-8187-a038889cf94f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:30.683410 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd78e28-a7f1-4f8f-8187-a038889cf94f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:30.691417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 36/74] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:30.692597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e711b3b-1c1e-40a1-9c1c-46706c68e73c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.692894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e711b3b-1c1e-40a1-9c1c-46706c68e73c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.693181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e711b3b-1c1e-40a1-9c1c-46706c68e73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.705708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:30.709367 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8e711b3b-1c1e-40a1-9c1c-46706c68e73c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:30.710593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8e711b3b-1c1e-40a1-9c1c-46706c68e73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:30.721379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51c6c7-6b3b-49f3-bb92-da92830ac30d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:30.755471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7d858-cb0a-4c64-8ac5-d0ba5550a24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:30.756354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7d858-cb0a-4c64-8ac5-d0ba5550a24f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.756644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7d858-cb0a-4c64-8ac5-d0ba5550a24f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.756931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7d858-cb0a-4c64-8ac5-d0ba5550a24f None admin] PATH_INFO: `/v3/users/d68b999da6f749cd98aa5b80062e64f3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.760416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:30.772715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7d858-cb0a-4c64-8ac5-d0ba5550a24f None admin] RBAC: Authorizing `identity:get_user(user_id=d68b999da6f749cd98aa5b80062e64f3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:30.774165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7d858-cb0a-4c64-8ac5-d0ba5550a24f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:30.775856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 39/75] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:30 2022] GET /identity/v3/users/d68b999da6f749cd98aa5b80062e64f3 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:30.776704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7d858-cb0a-4c64-8ac5-d0ba5550a24f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:30.780439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 37/76] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:30.782818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e711b3b-1c1e-40a1-9c1c-46706c68e73c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:30.881256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7d858-cb0a-4c64-8ac5-d0ba5550a24f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:30.883083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e711b3b-1c1e-40a1-9c1c-46706c68e73c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:30.920242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f82ec3-7b8c-4ea4-b4f6-8423dddb5f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:30.921072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f82ec3-7b8c-4ea4-b4f6-8423dddb5f85 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.921360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f82ec3-7b8c-4ea4-b4f6-8423dddb5f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.921639 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f82ec3-7b8c-4ea4-b4f6-8423dddb5f85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.924011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66385aff-e3cf-49e9-b112-aca8db3b8634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:30.925393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66385aff-e3cf-49e9-b112-aca8db3b8634 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:30.925685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66385aff-e3cf-49e9-b112-aca8db3b8634 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:30.926036 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66385aff-e3cf-49e9-b112-aca8db3b8634 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:30.932219 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f82ec3-7b8c-4ea4-b4f6-8423dddb5f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:30.937606 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66385aff-e3cf-49e9-b112-aca8db3b8634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c0f82ec3-7b8c-4ea4-b4f6-8423dddb5f85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:30.940270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:30.941762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 40/77] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:16:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:30.947817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f82ec3-7b8c-4ea4-b4f6-8423dddb5f85 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-66385aff-e3cf-49e9-b112-aca8db3b8634 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:30.948854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:30.949602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 38/78] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:30.950363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66385aff-e3cf-49e9-b112-aca8db3b8634 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:30.951392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:31.003790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:31.063863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66385aff-e3cf-49e9-b112-aca8db3b8634 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:31.069295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f82ec3-7b8c-4ea4-b4f6-8423dddb5f85 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:31.098312 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf575fd-3e88-42bc-a400-786bebfdb9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:31.098975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf575fd-3e88-42bc-a400-786bebfdb9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:31.099259 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf575fd-3e88-42bc-a400-786bebfdb9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:31.099543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf575fd-3e88-42bc-a400-786bebfdb9e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:31.106385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bc1c2-28b7-4322-aacb-2d276cef5102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:31.107346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bc1c2-28b7-4322-aacb-2d276cef5102 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:31.107641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bc1c2-28b7-4322-aacb-2d276cef5102 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:31.107924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bc1c2-28b7-4322-aacb-2d276cef5102 None admin] PATH_INFO: `/v3/users/a1033721c1da485882816b21d7d01a42` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:31.109151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bc1c2-28b7-4322-aacb-2d276cef5102 None admin] RBAC: Authorizing `identity:get_user(user_id=a1033721c1da485882816b21d7d01a42)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:31.110690 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bc1c2-28b7-4322-aacb-2d276cef5102 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:31.112888 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 41/79] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:30 2022] GET /identity/v3/users/a1033721c1da485882816b21d7d01a42 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:31.117846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bc1c2-28b7-4322-aacb-2d276cef5102 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:31.179137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf575fd-3e88-42bc-a400-786bebfdb9e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:31.181554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf575fd-3e88-42bc-a400-786bebfdb9e9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:31.199419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:31.226699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bc1c2-28b7-4322-aacb-2d276cef5102 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:31.232517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:31.260086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c0c80-d6a1-4b63-8d0a-798aa3460e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:31.260886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c0c80-d6a1-4b63-8d0a-798aa3460e29 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:31.261174 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c0c80-d6a1-4b63-8d0a-798aa3460e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:31.261456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c0c80-d6a1-4b63-8d0a-798aa3460e29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:31.262710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c0c80-d6a1-4b63-8d0a-798aa3460e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:31.264176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c0c80-d6a1-4b63-8d0a-798aa3460e29 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:31.265991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 42/80] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:31.266613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c0c80-d6a1-4b63-8d0a-798aa3460e29 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:31.290456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 39/81] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:31.291304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf575fd-3e88-42bc-a400-786bebfdb9e9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:31.371464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c0c80-d6a1-4b63-8d0a-798aa3460e29 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:31.388607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf575fd-3e88-42bc-a400-786bebfdb9e9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:31.407306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d359c-c0a8-4de0-acb1-4d1603b5b843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:31.407984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d359c-c0a8-4de0-acb1-4d1603b5b843 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:31.408275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d359c-c0a8-4de0-acb1-4d1603b5b843 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:31.408559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d359c-c0a8-4de0-acb1-4d1603b5b843 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:31.409899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d359c-c0a8-4de0-acb1-4d1603b5b843 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:31.410247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d359c-c0a8-4de0-acb1-4d1603b5b843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:31.411641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d359c-c0a8-4de0-acb1-4d1603b5b843 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:31.419890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 43/82] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:16:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:31.420676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d359c-c0a8-4de0-acb1-4d1603b5b843 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:31.423159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:31.424139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdd6da-eb76-4d07-a27b-76bff063e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:31.425275 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdd6da-eb76-4d07-a27b-76bff063e032 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:31.425592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdd6da-eb76-4d07-a27b-76bff063e032 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:31.425878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdd6da-eb76-4d07-a27b-76bff063e032 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:31.426675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbdd6da-eb76-4d07-a27b-76bff063e032 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:31.426997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbdd6da-eb76-4d07-a27b-76bff063e032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:31.428399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbdd6da-eb76-4d07-a27b-76bff063e032 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:31.436612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 40/83] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:31.437384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdd6da-eb76-4d07-a27b-76bff063e032 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:31.448063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:31.523231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d359c-c0a8-4de0-acb1-4d1603b5b843 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:31.538813 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdd6da-eb76-4d07-a27b-76bff063e032 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:31.560815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf43508-13ba-47df-a8d7-733d957adbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:31.561574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf43508-13ba-47df-a8d7-733d957adbed None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:31.561879 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf43508-13ba-47df-a8d7-733d957adbed None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:31.562268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf43508-13ba-47df-a8d7-733d957adbed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:31.574292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e0ad7-f5c0-4cf8-94bc-1dbff7e191c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:31.574999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e0ad7-f5c0-4cf8-94bc-1dbff7e191c3 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:31.575288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e0ad7-f5c0-4cf8-94bc-1dbff7e191c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:31.575574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e0ad7-f5c0-4cf8-94bc-1dbff7e191c3 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:31.576295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e0ad7-f5c0-4cf8-94bc-1dbff7e191c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:31.577739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e0ad7-f5c0-4cf8-94bc-1dbff7e191c3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:31.597175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 41/84] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:16:31 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:31.598028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e0ad7-f5c0-4cf8-94bc-1dbff7e191c3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:31.638796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf43508-13ba-47df-a8d7-733d957adbed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:31.640227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf43508-13ba-47df-a8d7-733d957adbed None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:31.642502 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:31.683652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:31.730743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e0ad7-f5c0-4cf8-94bc-1dbff7e191c3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:31.772171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7b86-4ea3-4c99-9d5c-26d136b769ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:31.773059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7b86-4ea3-4c99-9d5c-26d136b769ae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:31.773348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7b86-4ea3-4c99-9d5c-26d136b769ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:31.773835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7b86-4ea3-4c99-9d5c-26d136b769ae None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:31.774988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d7b86-4ea3-4c99-9d5c-26d136b769ae None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:31.776481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d7b86-4ea3-4c99-9d5c-26d136b769ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:31.782541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 44/85] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:31.782541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf43508-13ba-47df-a8d7-733d957adbed None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:31.782752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 42/86] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:31 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:31.782752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7b86-4ea3-4c99-9d5c-26d136b769ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:31.902875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf43508-13ba-47df-a8d7-733d957adbed None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:31.904622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:31.917222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7b86-4ea3-4c99-9d5c-26d136b769ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:31.944331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad89d48-2747-4e98-9e0b-17584a57b9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:31.945461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad89d48-2747-4e98-9e0b-17584a57b9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:31.945892 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad89d48-2747-4e98-9e0b-17584a57b9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:31.946369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad89d48-2747-4e98-9e0b-17584a57b9b3 None admin] PATH_INFO: `/v3/users/d68b999da6f749cd98aa5b80062e64f3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:31.947740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad89d48-2747-4e98-9e0b-17584a57b9b3 None admin] RBAC: Authorizing `identity:get_user(user_id=d68b999da6f749cd98aa5b80062e64f3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:31.949352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad89d48-2747-4e98-9e0b-17584a57b9b3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:31.951174 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 45/87] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:31 2022] GET /identity/v3/users/d68b999da6f749cd98aa5b80062e64f3 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:31.952079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad89d48-2747-4e98-9e0b-17584a57b9b3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:31.955461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:31.961635 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fade617-bc33-4bbd-8e50-376caf69f7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:31.963245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fade617-bc33-4bbd-8e50-376caf69f7eb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:31.963685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fade617-bc33-4bbd-8e50-376caf69f7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:31.964110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fade617-bc33-4bbd-8e50-376caf69f7eb None admin] PATH_INFO: `/v3/users/d68b999da6f749cd98aa5b80062e64f3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:31.965520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fade617-bc33-4bbd-8e50-376caf69f7eb None admin] RBAC: Authorizing `identity:get_user(user_id=d68b999da6f749cd98aa5b80062e64f3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:31.967179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fade617-bc33-4bbd-8e50-376caf69f7eb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:31.968991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 43/88] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:31 2022] GET /identity/v3/users/d68b999da6f749cd98aa5b80062e64f3 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:31.969911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fade617-bc33-4bbd-8e50-376caf69f7eb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:32.081245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fade617-bc33-4bbd-8e50-376caf69f7eb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:32.082851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad89d48-2747-4e98-9e0b-17584a57b9b3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:32.121911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca59a67-dd19-4056-bab8-dcd781201d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:32.122881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca59a67-dd19-4056-bab8-dcd781201d4c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:32.123139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca59a67-dd19-4056-bab8-dcd781201d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:32.123425 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca59a67-dd19-4056-bab8-dcd781201d4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:32.124278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca59a67-dd19-4056-bab8-dcd781201d4c None admin] RBAC: Adding query filter params (role.id=d3c75d38368d47d799bf59a0f9c879ea, scope.project.id=2f47c32296eb4caabf57eb7403220ebf, user.id=d68b999da6f749cd98aa5b80062e64f3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:32.124602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca59a67-dd19-4056-bab8-dcd781201d4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:32.126090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca59a67-dd19-4056-bab8-dcd781201d4c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:32.127183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9aea-7f38-4b19-81b5-88a901874976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:32.129175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9aea-7f38-4b19-81b5-88a901874976 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:32.129713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9aea-7f38-4b19-81b5-88a901874976 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:32.130180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9aea-7f38-4b19-81b5-88a901874976 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:32.131540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf9aea-7f38-4b19-81b5-88a901874976 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:32.132501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 44/89] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:31 2022] GET /identity/v3/role_assignments?user.id=d68b999da6f749cd98aa5b80062e64f3&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:32.133303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca59a67-dd19-4056-bab8-dcd781201d4c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:32.133615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf9aea-7f38-4b19-81b5-88a901874976 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:32.135969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 46/90] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:31 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:32.137198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9aea-7f38-4b19-81b5-88a901874976 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:32.184643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:32.224066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:32.295923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9aea-7f38-4b19-81b5-88a901874976 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:32.309369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca59a67-dd19-4056-bab8-dcd781201d4c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:32.365821 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b5ad3-ebd3-4c7c-91b1-cbc1573d8f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:32.366853 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b5ad3-ebd3-4c7c-91b1-cbc1573d8f13 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:32.367146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b5ad3-ebd3-4c7c-91b1-cbc1573d8f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:32.367428 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b5ad3-ebd3-4c7c-91b1-cbc1573d8f13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:32.369204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe4520-5d1f-4bb8-ade3-d244a86d419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:32.369842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe4520-5d1f-4bb8-ade3-d244a86d419b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:32.370171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe4520-5d1f-4bb8-ade3-d244a86d419b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:32.370454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe4520-5d1f-4bb8-ade3-d244a86d419b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:32.451767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b5ad3-ebd3-4c7c-91b1-cbc1573d8f13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:32.453171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b5ad3-ebd3-4c7c-91b1-cbc1573d8f13 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:32.500487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:32.562905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe4520-5d1f-4bb8-ade3-d244a86d419b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:32.564791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe4520-5d1f-4bb8-ade3-d244a86d419b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:32.580107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 47/91] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:32.580107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b5ad3-ebd3-4c7c-91b1-cbc1573d8f13 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:32.702707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b5ad3-ebd3-4c7c-91b1-cbc1573d8f13 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:32.739459 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:32.755016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fe84b-4ad0-451a-aa81-c635203cfbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:32.755697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe84b-4ad0-451a-aa81-c635203cfbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:32.755984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe84b-4ad0-451a-aa81-c635203cfbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:32.756269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe84b-4ad0-451a-aa81-c635203cfbb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:32.757122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fe84b-4ad0-451a-aa81-c635203cfbb1 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=2f47c32296eb4caabf57eb7403220ebf, user.id=a1033721c1da485882816b21d7d01a42) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:32.757442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fe84b-4ad0-451a-aa81-c635203cfbb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:32.758924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fe84b-4ad0-451a-aa81-c635203cfbb1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:32.772519 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 48/92] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:32 2022] GET /identity/v3/role_assignments?user.id=a1033721c1da485882816b21d7d01a42&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:32.773678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fe84b-4ad0-451a-aa81-c635203cfbb1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:32.780890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:32.804124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 45/93] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:32.811109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe4520-5d1f-4bb8-ade3-d244a86d419b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:32.905991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fe84b-4ad0-451a-aa81-c635203cfbb1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:32.943236 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eed688-30ca-43d0-81e2-920eaed74531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:32.944119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eed688-30ca-43d0-81e2-920eaed74531 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:32.944412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eed688-30ca-43d0-81e2-920eaed74531 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:32.945070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eed688-30ca-43d0-81e2-920eaed74531 None admin] PATH_INFO: `/v3/projects/5278726ed9db4870bb01682315cbb3e6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:32.945910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eed688-30ca-43d0-81e2-920eaed74531 None admin] RBAC: Authorizing `identity:get_project(project_id=5278726ed9db4870bb01682315cbb3e6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:32.947436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eed688-30ca-43d0-81e2-920eaed74531 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:32.949228 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 49/94] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:32 2022] GET /identity/v3/projects/5278726ed9db4870bb01682315cbb3e6 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:32.956568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eed688-30ca-43d0-81e2-920eaed74531 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:32.993694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe4520-5d1f-4bb8-ade3-d244a86d419b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:33.074413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1277b-beab-47a2-9e93-49dc7e5f7762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:33.075302 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1277b-beab-47a2-9e93-49dc7e5f7762 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:33.075589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1277b-beab-47a2-9e93-49dc7e5f7762 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:33.075869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1277b-beab-47a2-9e93-49dc7e5f7762 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:33.082279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1277b-beab-47a2-9e93-49dc7e5f7762 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:33.083780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1277b-beab-47a2-9e93-49dc7e5f7762 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:33.090123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 46/95] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:32 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:33.091878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1277b-beab-47a2-9e93-49dc7e5f7762 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:33.099212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:33.141154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:33.160901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eed688-30ca-43d0-81e2-920eaed74531 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:33.226516 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8762581-5283-4457-89b7-b27b37dc820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:33.227559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8762581-5283-4457-89b7-b27b37dc820b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:33.228121 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8762581-5283-4457-89b7-b27b37dc820b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:33.230360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8762581-5283-4457-89b7-b27b37dc820b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:33.231245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8762581-5283-4457-89b7-b27b37dc820b None admin] RBAC: Adding query filter params (role.id=d3c75d38368d47d799bf59a0f9c879ea, scope.project.id=5278726ed9db4870bb01682315cbb3e6, user.id=d68b999da6f749cd98aa5b80062e64f3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:33.231570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8762581-5283-4457-89b7-b27b37dc820b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:33.233042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8762581-5283-4457-89b7-b27b37dc820b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:33.245965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 50/96] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:32 2022] GET /identity/v3/role_assignments?user.id=d68b999da6f749cd98aa5b80062e64f3&scope.project.id=5278726ed9db4870bb01682315cbb3e6&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:33.270930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1277b-beab-47a2-9e93-49dc7e5f7762 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:33.331166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be914dbd-44b1-41b3-82c0-3cf300e28ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:33.332222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be914dbd-44b1-41b3-82c0-3cf300e28ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:33.332702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be914dbd-44b1-41b3-82c0-3cf300e28ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:33.333197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be914dbd-44b1-41b3-82c0-3cf300e28ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:33.334298 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be914dbd-44b1-41b3-82c0-3cf300e28ea0 None admin] RBAC: Adding query filter params (role.id=d534ab3d0a904f0babeb90d2679d1b86, scope.project.id=2f47c32296eb4caabf57eb7403220ebf, user.id=d68b999da6f749cd98aa5b80062e64f3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:33.334812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be914dbd-44b1-41b3-82c0-3cf300e28ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:33.336452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be914dbd-44b1-41b3-82c0-3cf300e28ea0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:33.342828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 47/97] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:33 2022] GET /identity/v3/role_assignments?user.id=d68b999da6f749cd98aa5b80062e64f3&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=d534ab3d0a904f0babeb90d2679d1b86 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:34.559344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a845a-fd95-4f30-9661-2ef0d07eb66c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:34.559853 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a845a-fd95-4f30-9661-2ef0d07eb66c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:34.559926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a845a-fd95-4f30-9661-2ef0d07eb66c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:34.561267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 51/98] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:34.567201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd435f-21ec-4b7c-b6dd-19a981cfe446 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:34.567600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd435f-21ec-4b7c-b6dd-19a981cfe446 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:34.567765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd435f-21ec-4b7c-b6dd-19a981cfe446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:34.590270 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cddd435f-21ec-4b7c-b6dd-19a981cfe446 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:34.591453 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cddd435f-21ec-4b7c-b6dd-19a981cfe446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:34.620860 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760ed80-f4bd-4862-b468-124a6e5ebaad None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:34.621162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760ed80-f4bd-4862-b468-124a6e5ebaad None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:34.621425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760ed80-f4bd-4862-b468-124a6e5ebaad None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:34.623104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 52/99] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:34.631746 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8637f0a-2d60-41ef-b5df-735779d32ef9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:34.632031 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8637f0a-2d60-41ef-b5df-735779d32ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:34.632312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8637f0a-2d60-41ef-b5df-735779d32ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:34.655681 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a8637f0a-2d60-41ef-b5df-735779d32ef9 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:34.656882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a8637f0a-2d60-41ef-b5df-735779d32ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:34.758212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 48/100] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:16:34.778076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19f8a7-f707-4bbd-bd58-9ab4fb2dae44 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:34.778403 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19f8a7-f707-4bbd-bd58-9ab4fb2dae44 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:34.778643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19f8a7-f707-4bbd-bd58-9ab4fb2dae44 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:34.780376 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 49/101] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:16:34.786611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033e893-8777-4eee-9974-adeac83c6d16 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:34.786890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033e893-8777-4eee-9974-adeac83c6d16 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:34.787181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033e893-8777-4eee-9974-adeac83c6d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:34.808092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d033e893-8777-4eee-9974-adeac83c6d16 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:34.809483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d033e893-8777-4eee-9974-adeac83c6d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:34.810552 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 53/102] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:16:34.811607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8637f0a-2d60-41ef-b5df-735779d32ef9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:34.856702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:34.942765 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:34.949127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 50/103] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:34.950771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033e893-8777-4eee-9974-adeac83c6d16 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:35.019098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8637f0a-2d60-41ef-b5df-735779d32ef9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:35.098181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033e893-8777-4eee-9974-adeac83c6d16 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:35.134818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5e78c-a997-4b8a-9e42-464c83804529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:35.135662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e78c-a997-4b8a-9e42-464c83804529 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:35.135993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e78c-a997-4b8a-9e42-464c83804529 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:35.136277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e78c-a997-4b8a-9e42-464c83804529 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:35.137454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5e78c-a997-4b8a-9e42-464c83804529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:35.139183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5e78c-a997-4b8a-9e42-464c83804529 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:35.140811 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 51/104] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:35.142534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5e78c-a997-4b8a-9e42-464c83804529 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:35.150674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26258435-200f-4650-bbab-51f35945f2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:35.151494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258435-200f-4650-bbab-51f35945f2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:35.151778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258435-200f-4650-bbab-51f35945f2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:35.152058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258435-200f-4650-bbab-51f35945f2a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:35.171092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:35.191732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26258435-200f-4650-bbab-51f35945f2a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-26258435-200f-4650-bbab-51f35945f2a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:35.208256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:35.209191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:35.209989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 54/105] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:35.210631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26258435-200f-4650-bbab-51f35945f2a3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:35.256578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5e78c-a997-4b8a-9e42-464c83804529 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:35.296101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36762b-c6f8-464a-b8cd-3d7fd6b4fb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:35.297051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36762b-c6f8-464a-b8cd-3d7fd6b4fb71 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:35.297602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36762b-c6f8-464a-b8cd-3d7fd6b4fb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:35.297892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36762b-c6f8-464a-b8cd-3d7fd6b4fb71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:35.309703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36762b-c6f8-464a-b8cd-3d7fd6b4fb71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ff36762b-c6f8-464a-b8cd-3d7fd6b4fb71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:35.318709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:35.319540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:35.320322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 52/106] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:35.326194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36762b-c6f8-464a-b8cd-3d7fd6b4fb71 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:35.428306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:35.446878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36762b-c6f8-464a-b8cd-3d7fd6b4fb71 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:35.457291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26258435-200f-4650-bbab-51f35945f2a3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:35.485636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1ebdd-695f-4d1c-b2cd-4328a97917aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:35.486396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1ebdd-695f-4d1c-b2cd-4328a97917aa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:35.486689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1ebdd-695f-4d1c-b2cd-4328a97917aa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:35.486970 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1ebdd-695f-4d1c-b2cd-4328a97917aa None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:35.487781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1ebdd-695f-4d1c-b2cd-4328a97917aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:35.488097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1ebdd-695f-4d1c-b2cd-4328a97917aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:35.490108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1ebdd-695f-4d1c-b2cd-4328a97917aa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:35.501961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 53/107] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:35.501961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1ebdd-695f-4d1c-b2cd-4328a97917aa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:35.510945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:35.538647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94523e13-2ee9-453f-a358-6fcde79076d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:35.539332 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94523e13-2ee9-453f-a358-6fcde79076d9 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:35.539623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94523e13-2ee9-453f-a358-6fcde79076d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:35.539909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94523e13-2ee9-453f-a358-6fcde79076d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:35.540624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94523e13-2ee9-453f-a358-6fcde79076d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:35.542381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94523e13-2ee9-453f-a358-6fcde79076d9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:35.618740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 55/108] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:16:35 2022] POST /identity/v3/projects => generated 302 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:35.619571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94523e13-2ee9-453f-a358-6fcde79076d9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:35.623538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1ebdd-695f-4d1c-b2cd-4328a97917aa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:35.671392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65902164-8d0b-4aa3-bf24-df81dff4dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:35.672926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65902164-8d0b-4aa3-bf24-df81dff4dec3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:35.673467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65902164-8d0b-4aa3-bf24-df81dff4dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:35.673925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65902164-8d0b-4aa3-bf24-df81dff4dec3 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:35.674942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65902164-8d0b-4aa3-bf24-df81dff4dec3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:35.675458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65902164-8d0b-4aa3-bf24-df81dff4dec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:35.677145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65902164-8d0b-4aa3-bf24-df81dff4dec3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:35.689652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 54/109] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:35.697905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65902164-8d0b-4aa3-bf24-df81dff4dec3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:35.734364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:35.826190 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65902164-8d0b-4aa3-bf24-df81dff4dec3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:35.868284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a5b7a-cee1-4681-be03-1b086de9c2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:35.869634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a5b7a-cee1-4681-be03-1b086de9c2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:35.870102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a5b7a-cee1-4681-be03-1b086de9c2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:35.870525 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a5b7a-cee1-4681-be03-1b086de9c2f0 None admin] PATH_INFO: `/v3/users/d68b999da6f749cd98aa5b80062e64f3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:35.872319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a5b7a-cee1-4681-be03-1b086de9c2f0 None admin] RBAC: Authorizing `identity:get_user(user_id=d68b999da6f749cd98aa5b80062e64f3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:35.873805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a5b7a-cee1-4681-be03-1b086de9c2f0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:35.876016 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 55/110] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:35 2022] GET /identity/v3/users/d68b999da6f749cd98aa5b80062e64f3 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:35.877385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41cfd-c73a-4790-ba8f-57b2981ba52e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:35.877677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41cfd-c73a-4790-ba8f-57b2981ba52e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:35.877973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41cfd-c73a-4790-ba8f-57b2981ba52e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:35.879374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 56/111] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:35.880436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94523e13-2ee9-453f-a358-6fcde79076d9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:35.885457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba41cfd-c73a-4790-ba8f-57b2981ba52e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:35.932747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf884ba-b2e0-4e4f-af5b-a1359898326b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:35.933997 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf884ba-b2e0-4e4f-af5b-a1359898326b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:35.935026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf884ba-b2e0-4e4f-af5b-a1359898326b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:35.935507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf884ba-b2e0-4e4f-af5b-a1359898326b None admin] PATH_INFO: `/v3/users/148723ffdfc74b8d87150f004cdfbd01` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:35.956251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf884ba-b2e0-4e4f-af5b-a1359898326b None admin] RBAC: Authorizing `identity:get_user(user_id=148723ffdfc74b8d87150f004cdfbd01)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:35.958616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf884ba-b2e0-4e4f-af5b-a1359898326b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:35.961574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 56/112] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:35 2022] GET /identity/v3/users/148723ffdfc74b8d87150f004cdfbd01 => generated 289 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:35.963435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da33a-bf90-4b43-a903-829db28bf0f8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:35.964333 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da33a-bf90-4b43-a903-829db28bf0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:35.965478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da33a-bf90-4b43-a903-829db28bf0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:35.985905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b11da33a-bf90-4b43-a903-829db28bf0f8 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:35.990173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b11da33a-bf90-4b43-a903-829db28bf0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:35.996652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:36.020527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:36.026108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba41cfd-c73a-4790-ba8f-57b2981ba52e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:36.066940 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398df24c-490f-4fe1-99a3-3dc47908f62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:36.068708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398df24c-490f-4fe1-99a3-3dc47908f62f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:36.068708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398df24c-490f-4fe1-99a3-3dc47908f62f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:36.068708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398df24c-490f-4fe1-99a3-3dc47908f62f None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:36.070419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398df24c-490f-4fe1-99a3-3dc47908f62f None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:36.071875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398df24c-490f-4fe1-99a3-3dc47908f62f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:36.073862 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 57/113] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:35 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:36.074752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398df24c-490f-4fe1-99a3-3dc47908f62f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:36.076957 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 57/114] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:36.079028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dc182-4618-42ed-ab6e-c0f5b077d4d2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:36.079341 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dc182-4618-42ed-ab6e-c0f5b077d4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:36.079630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dc182-4618-42ed-ab6e-c0f5b077d4d2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:36.086033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 58/115] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:36.086787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9dc182-4618-42ed-ab6e-c0f5b077d4d2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:36.206838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398df24c-490f-4fe1-99a3-3dc47908f62f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:36.218229 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9dc182-4618-42ed-ab6e-c0f5b077d4d2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:36.250118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608eb2e-467e-4ea3-9e66-665a8f01106d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:36.251745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608eb2e-467e-4ea3-9e66-665a8f01106d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:36.252041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608eb2e-467e-4ea3-9e66-665a8f01106d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:36.252329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608eb2e-467e-4ea3-9e66-665a8f01106d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:36.268949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430eede-0a9f-415c-b411-d84e6522dff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:36.270694 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430eede-0a9f-415c-b411-d84e6522dff0 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:36.271081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430eede-0a9f-415c-b411-d84e6522dff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:36.271459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430eede-0a9f-415c-b411-d84e6522dff0 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf/users/d68b999da6f749cd98aa5b80062e64f3/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:36.272037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:36.274647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430eede-0a9f-415c-b411-d84e6522dff0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f47c32296eb4caabf57eb7403220ebf, user_id=d68b999da6f749cd98aa5b80062e64f3, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:36.276538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430eede-0a9f-415c-b411-d84e6522dff0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:36.295953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 59/116] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:16:36 2022] PUT /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf/users/d68b999da6f749cd98aa5b80062e64f3/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:16:36.298062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd15b4-ca2f-4fd9-905f-125b2a2f288d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:36.298437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd15b4-ca2f-4fd9-905f-125b2a2f288d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:36.298794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd15b4-ca2f-4fd9-905f-125b2a2f288d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:36.319202 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b8cd15b4-ca2f-4fd9-905f-125b2a2f288d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:36.320730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b8cd15b4-ca2f-4fd9-905f-125b2a2f288d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:36.344084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:36.376538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608eb2e-467e-4ea3-9e66-665a8f01106d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:36.378209 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608eb2e-467e-4ea3-9e66-665a8f01106d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:36.411398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 60/117] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:36.412095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd15b4-ca2f-4fd9-905f-125b2a2f288d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:36.503216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 58/118] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:36.504933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15d5b-0214-49fe-bb1b-2ad932db5a5d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:36.505298 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15d5b-0214-49fe-bb1b-2ad932db5a5d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:36.505586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15d5b-0214-49fe-bb1b-2ad932db5a5d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:36.507189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 59/119] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:36.513817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b15d5b-0214-49fe-bb1b-2ad932db5a5d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:36.550360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd15b4-ca2f-4fd9-905f-125b2a2f288d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:36.579459 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:36.597351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77a71-1f39-4397-86d6-ee30248f0a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:36.598446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77a71-1f39-4397-86d6-ee30248f0a09 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:36.598741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77a71-1f39-4397-86d6-ee30248f0a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:36.599424 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77a71-1f39-4397-86d6-ee30248f0a09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:36.614136 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a77a71-1f39-4397-86d6-ee30248f0a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-22a77a71-1f39-4397-86d6-ee30248f0a09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:36.623782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:36.625040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:36.626305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 61/120] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:36.627944 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77a71-1f39-4397-86d6-ee30248f0a09 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:36.649309 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:36.649849 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b15d5b-0214-49fe-bb1b-2ad932db5a5d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:36.697876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad8969-7d5f-4cae-96df-8ba3c35f85ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:36.699411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad8969-7d5f-4cae-96df-8ba3c35f85ed None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:36.700027 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad8969-7d5f-4cae-96df-8ba3c35f85ed None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:36.700384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad8969-7d5f-4cae-96df-8ba3c35f85ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:36.718078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad8969-7d5f-4cae-96df-8ba3c35f85ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-21ad8969-7d5f-4cae-96df-8ba3c35f85ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:36.729587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:36.731175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:36.732809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 60/121] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:36.736699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf666a7-76c1-4f4a-a2ce-d1f5aa394040 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:36.736699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf666a7-76c1-4f4a-a2ce-d1f5aa394040 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:36.736699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf666a7-76c1-4f4a-a2ce-d1f5aa394040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:36.755132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ebf666a7-76c1-4f4a-a2ce-d1f5aa394040 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:36.756300 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ebf666a7-76c1-4f4a-a2ce-d1f5aa394040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:36.765002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77a71-1f39-4397-86d6-ee30248f0a09 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:36.804988 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb57fc-5982-45c9-95b8-35a289e99a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:36.806005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb57fc-5982-45c9-95b8-35a289e99a20 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:36.806305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb57fc-5982-45c9-95b8-35a289e99a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:36.806594 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb57fc-5982-45c9-95b8-35a289e99a20 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:36.807855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb57fc-5982-45c9-95b8-35a289e99a20 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:36.809439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb57fc-5982-45c9-95b8-35a289e99a20 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:36.816750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 62/122] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:36 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:36.817683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb57fc-5982-45c9-95b8-35a289e99a20 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:36.832036 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 61/123] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:36.838801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf666a7-76c1-4f4a-a2ce-d1f5aa394040 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:36.891995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:36.914253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:36.942680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb57fc-5982-45c9-95b8-35a289e99a20 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:36.972619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf666a7-76c1-4f4a-a2ce-d1f5aa394040 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:36.992702 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6011c-a17c-4ab5-afbd-bf98e1a5ca39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:36.993399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6011c-a17c-4ab5-afbd-bf98e1a5ca39 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:36.993689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6011c-a17c-4ab5-afbd-bf98e1a5ca39 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:36.994001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6011c-a17c-4ab5-afbd-bf98e1a5ca39 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:36.994776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6011c-a17c-4ab5-afbd-bf98e1a5ca39 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:36.995625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6011c-a17c-4ab5-afbd-bf98e1a5ca39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:36.997326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6011c-a17c-4ab5-afbd-bf98e1a5ca39 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:37.009150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 63/124] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:37.009918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6011c-a17c-4ab5-afbd-bf98e1a5ca39 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:37.010621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967be95c-99b8-41f0-982f-64e04ec67676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:37.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967be95c-99b8-41f0-982f-64e04ec67676 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:37.011919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967be95c-99b8-41f0-982f-64e04ec67676 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:37.012196 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967be95c-99b8-41f0-982f-64e04ec67676 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:37.013067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967be95c-99b8-41f0-982f-64e04ec67676 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:37.013767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967be95c-99b8-41f0-982f-64e04ec67676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:37.015255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967be95c-99b8-41f0-982f-64e04ec67676 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:37.026648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 62/125] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:37.027337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967be95c-99b8-41f0-982f-64e04ec67676 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:37.134836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6011c-a17c-4ab5-afbd-bf98e1a5ca39 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:37.156635 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:37.166806 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967be95c-99b8-41f0-982f-64e04ec67676 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:37.173693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cddad-e29e-4b67-9a9f-149274e5ca24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:37.174724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cddad-e29e-4b67-9a9f-149274e5ca24 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:37.175224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cddad-e29e-4b67-9a9f-149274e5ca24 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:37.175712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cddad-e29e-4b67-9a9f-149274e5ca24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:37.176753 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cddad-e29e-4b67-9a9f-149274e5ca24 None admin] RBAC: Adding query filter params (role.id=a889423e7cb541dc99e90f4293116033, scope.project.id=2f47c32296eb4caabf57eb7403220ebf, user.id=148723ffdfc74b8d87150f004cdfbd01) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:37.177288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cddad-e29e-4b67-9a9f-149274e5ca24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:37.178980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cddad-e29e-4b67-9a9f-149274e5ca24 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:37.184950 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 64/126] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:37 2022] GET /identity/v3/role_assignments?user.id=148723ffdfc74b8d87150f004cdfbd01&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=a889423e7cb541dc99e90f4293116033 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:37.186328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cddad-e29e-4b67-9a9f-149274e5ca24 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:37.199133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:37.221671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d567bc5-6827-49a3-826f-984075ee912d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:37.222974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d567bc5-6827-49a3-826f-984075ee912d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:37.223784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d567bc5-6827-49a3-826f-984075ee912d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:37.225269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d567bc5-6827-49a3-826f-984075ee912d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:37.249985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d567bc5-6827-49a3-826f-984075ee912d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2d567bc5-6827-49a3-826f-984075ee912d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:37.263692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:37.264631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:37.265974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:37.276218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 63/127] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:16:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:37.276218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d567bc5-6827-49a3-826f-984075ee912d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:37.400331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d567bc5-6827-49a3-826f-984075ee912d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:37.413817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cddad-e29e-4b67-9a9f-149274e5ca24 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:37.445995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb06f8-3053-49b7-8c6d-76cb9e181d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:37.447147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb06f8-3053-49b7-8c6d-76cb9e181d76 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:37.447439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb06f8-3053-49b7-8c6d-76cb9e181d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:37.447725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb06f8-3053-49b7-8c6d-76cb9e181d76 None admin] PATH_INFO: `/v3/users/a1033721c1da485882816b21d7d01a42` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:37.450509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb06f8-3053-49b7-8c6d-76cb9e181d76 None admin] RBAC: Authorizing `identity:get_user(user_id=a1033721c1da485882816b21d7d01a42)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:37.452031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb06f8-3053-49b7-8c6d-76cb9e181d76 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:37.454893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 64/128] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:37 2022] GET /identity/v3/users/a1033721c1da485882816b21d7d01a42 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:37.455807 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb06f8-3053-49b7-8c6d-76cb9e181d76 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:37.456366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3979b-ad91-437c-b888-f2363b355f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:37.457547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3979b-ad91-437c-b888-f2363b355f77 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:37.458071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3979b-ad91-437c-b888-f2363b355f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:37.458557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3979b-ad91-437c-b888-f2363b355f77 None admin] PATH_INFO: `/v3/users/d68b999da6f749cd98aa5b80062e64f3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:37.461421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3979b-ad91-437c-b888-f2363b355f77 None admin] RBAC: Authorizing `identity:get_user(user_id=d68b999da6f749cd98aa5b80062e64f3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:37.463125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3979b-ad91-437c-b888-f2363b355f77 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:37.465314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 65/129] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:37 2022] GET /identity/v3/users/d68b999da6f749cd98aa5b80062e64f3 => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:37.466825 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3979b-ad91-437c-b888-f2363b355f77 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:37.492497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:37.584768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb06f8-3053-49b7-8c6d-76cb9e181d76 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:37.629669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00009dcf-4d2e-40b5-b383-90d5a88e02bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:37.630819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00009dcf-4d2e-40b5-b383-90d5a88e02bb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:37.630819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00009dcf-4d2e-40b5-b383-90d5a88e02bb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:37.631092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00009dcf-4d2e-40b5-b383-90d5a88e02bb None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:37.631848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00009dcf-4d2e-40b5-b383-90d5a88e02bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:37.632167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00009dcf-4d2e-40b5-b383-90d5a88e02bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:37.633575 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00009dcf-4d2e-40b5-b383-90d5a88e02bb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:37.647710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:37.657922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 65/130] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:16:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:37.658829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00009dcf-4d2e-40b5-b383-90d5a88e02bb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:37.676360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3979b-ad91-437c-b888-f2363b355f77 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:37.718336 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8297b-313a-4127-82fc-5224d7f9dbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:37.719255 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8297b-313a-4127-82fc-5224d7f9dbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:37.719538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8297b-313a-4127-82fc-5224d7f9dbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:37.719816 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8297b-313a-4127-82fc-5224d7f9dbd0 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:37.724457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8297b-313a-4127-82fc-5224d7f9dbd0 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:37.725951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8297b-313a-4127-82fc-5224d7f9dbd0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:37.727721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 66/131] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:37 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:37.728499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8297b-313a-4127-82fc-5224d7f9dbd0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:37.847365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:37.884723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00009dcf-4d2e-40b5-b383-90d5a88e02bb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:37.890246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8297b-313a-4127-82fc-5224d7f9dbd0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:37.941071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f7eb2-334d-4000-92cd-3d24b75919f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:37.944260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f7eb2-334d-4000-92cd-3d24b75919f0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:37.944548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f7eb2-334d-4000-92cd-3d24b75919f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:37.944831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f7eb2-334d-4000-92cd-3d24b75919f0 None admin] PATH_INFO: `/v3/users/d68b999da6f749cd98aa5b80062e64f3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:37.950736 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f7eb2-334d-4000-92cd-3d24b75919f0 None admin] RBAC: Authorizing `identity:get_user(user_id=d68b999da6f749cd98aa5b80062e64f3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:37.952247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f7eb2-334d-4000-92cd-3d24b75919f0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:37.954138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 67/132] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:37 2022] GET /identity/v3/users/d68b999da6f749cd98aa5b80062e64f3 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:37.954920 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f7eb2-334d-4000-92cd-3d24b75919f0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:37.964049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236d7ad-641f-4cc6-82bc-a687f46a410f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:37.964941 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236d7ad-641f-4cc6-82bc-a687f46a410f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:37.965229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236d7ad-641f-4cc6-82bc-a687f46a410f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:37.965507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236d7ad-641f-4cc6-82bc-a687f46a410f None admin] PATH_INFO: `/v3/projects/5278726ed9db4870bb01682315cbb3e6` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:37.970677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236d7ad-641f-4cc6-82bc-a687f46a410f None admin] RBAC: Authorizing `identity:get_project(project_id=5278726ed9db4870bb01682315cbb3e6)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:37.972150 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236d7ad-641f-4cc6-82bc-a687f46a410f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:37.974046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 66/133] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:37 2022] GET /identity/v3/projects/5278726ed9db4870bb01682315cbb3e6 => generated 312 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:37.974723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236d7ad-641f-4cc6-82bc-a687f46a410f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:37.977024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:38.095753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236d7ad-641f-4cc6-82bc-a687f46a410f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:38.138665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d362970-ffc5-4dbb-a096-b7e4f38a439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:38.140426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d362970-ffc5-4dbb-a096-b7e4f38a439a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:38.140921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d362970-ffc5-4dbb-a096-b7e4f38a439a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:38.141407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d362970-ffc5-4dbb-a096-b7e4f38a439a None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:38.143441 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d362970-ffc5-4dbb-a096-b7e4f38a439a None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:38.145407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d362970-ffc5-4dbb-a096-b7e4f38a439a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:38.148314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 67/134] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:37 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:38.149975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d362970-ffc5-4dbb-a096-b7e4f38a439a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:38.156569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f7eb2-334d-4000-92cd-3d24b75919f0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:38.186478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:38.200336 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2efb9-4ae9-4b21-9f94-a4ea07e56a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:38.201259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2efb9-4ae9-4b21-9f94-a4ea07e56a98 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:38.201968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2efb9-4ae9-4b21-9f94-a4ea07e56a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:38.202274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2efb9-4ae9-4b21-9f94-a4ea07e56a98 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf/users/a1033721c1da485882816b21d7d01a42/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:38.205212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2efb9-4ae9-4b21-9f94-a4ea07e56a98 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f47c32296eb4caabf57eb7403220ebf, user_id=a1033721c1da485882816b21d7d01a42, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:38.207059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2efb9-4ae9-4b21-9f94-a4ea07e56a98 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:38.225208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 68/135] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:16:37 2022] PUT /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf/users/a1033721c1da485882816b21d7d01a42/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:16:38.231201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb067d2-a2db-4c6d-8e1b-e98dd7445ed8 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:38.231502 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb067d2-a2db-4c6d-8e1b-e98dd7445ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:38.231795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb067d2-a2db-4c6d-8e1b-e98dd7445ed8 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:38.233299 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 69/136] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:38.238500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb067d2-a2db-4c6d-8e1b-e98dd7445ed8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:38.283532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d362970-ffc5-4dbb-a096-b7e4f38a439a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:38.333284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773f5c0-b8a2-47b2-955a-49b9052c0ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:38.334635 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773f5c0-b8a2-47b2-955a-49b9052c0ff3 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:38.335140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773f5c0-b8a2-47b2-955a-49b9052c0ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:38.336548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773f5c0-b8a2-47b2-955a-49b9052c0ff3 None admin] PATH_INFO: `/v3/projects/5278726ed9db4870bb01682315cbb3e6/users/d68b999da6f749cd98aa5b80062e64f3/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:38.342528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773f5c0-b8a2-47b2-955a-49b9052c0ff3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5278726ed9db4870bb01682315cbb3e6, user_id=d68b999da6f749cd98aa5b80062e64f3, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:38.344522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773f5c0-b8a2-47b2-955a-49b9052c0ff3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:38.373044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 68/137] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:16:38 2022] PUT /identity/v3/projects/5278726ed9db4870bb01682315cbb3e6/users/d68b999da6f749cd98aa5b80062e64f3/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:16:38.375316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d84b9-0097-4e51-981b-2bbd413ce1e1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:38.376804 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d84b9-0097-4e51-981b-2bbd413ce1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:38.377610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d84b9-0097-4e51-981b-2bbd413ce1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:38.398712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:38.419304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fd7d84b9-0097-4e51-981b-2bbd413ce1e1 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:38.420959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fd7d84b9-0097-4e51-981b-2bbd413ce1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:38.476896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb067d2-a2db-4c6d-8e1b-e98dd7445ed8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:38.518650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 69/138] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:38.543458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d84b9-0097-4e51-981b-2bbd413ce1e1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:38.551761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1aa4e-f83e-4c6d-8613-e7e6451c2dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:38.552680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1aa4e-f83e-4c6d-8613-e7e6451c2dc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:38.552964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1aa4e-f83e-4c6d-8613-e7e6451c2dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:38.553246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1aa4e-f83e-4c6d-8613-e7e6451c2dc2 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf/users/d68b999da6f749cd98aa5b80062e64f3/roles/d534ab3d0a904f0babeb90d2679d1b86` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:38.555636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1aa4e-f83e-4c6d-8613-e7e6451c2dc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f47c32296eb4caabf57eb7403220ebf, user_id=d68b999da6f749cd98aa5b80062e64f3, role_id=d534ab3d0a904f0babeb90d2679d1b86)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:38.557147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1aa4e-f83e-4c6d-8613-e7e6451c2dc2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:38.560904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:38.588203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 70/139] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:16:38 2022] PUT /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf/users/d68b999da6f749cd98aa5b80062e64f3/roles/d534ab3d0a904f0babeb90d2679d1b86 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:16:38.664934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d84b9-0097-4e51-981b-2bbd413ce1e1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:38.705139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3f103-dc16-4574-86cc-9b19a5a9b0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:38.705986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3f103-dc16-4574-86cc-9b19a5a9b0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:38.706277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3f103-dc16-4574-86cc-9b19a5a9b0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:38.706556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3f103-dc16-4574-86cc-9b19a5a9b0f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:38.707758 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3f103-dc16-4574-86cc-9b19a5a9b0f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:38.709228 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3f103-dc16-4574-86cc-9b19a5a9b0f4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:38.710907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 70/140] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:38.719580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1aa4e-f83e-4c6d-8613-e7e6451c2dc2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:38.730324 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa90be-8607-4d67-aa6c-e62bbe5ccbee None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:38.730650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa90be-8607-4d67-aa6c-e62bbe5ccbee None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:38.730899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa90be-8607-4d67-aa6c-e62bbe5ccbee None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:38.732482 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 71/141] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:16:38.738279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafb1be-7024-4537-9406-3b5132697a2e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:38.738568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafb1be-7024-4537-9406-3b5132697a2e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:38.738855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafb1be-7024-4537-9406-3b5132697a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:38.756797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ccafb1be-7024-4537-9406-3b5132697a2e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:38.758070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ccafb1be-7024-4537-9406-3b5132697a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:38.868619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 72/142] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:16:38.906667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafb1be-7024-4537-9406-3b5132697a2e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:38.923181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1aa4e-f83e-4c6d-8613-e7e6451c2dc2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:38.930072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:38.958323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:38.975658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e80dae-8667-4201-b7cc-7347b3f2643d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:38.976375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e80dae-8667-4201-b7cc-7347b3f2643d None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:38.976678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e80dae-8667-4201-b7cc-7347b3f2643d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:38.976972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e80dae-8667-4201-b7cc-7347b3f2643d None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:38.977722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e80dae-8667-4201-b7cc-7347b3f2643d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:38.979358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e80dae-8667-4201-b7cc-7347b3f2643d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:39.041767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 71/143] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:16:38 2022] POST /identity/v3/users => generated 283 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:39.072697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafb1be-7024-4537-9406-3b5132697a2e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:39.119744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e374239-f8af-45f2-a02f-4169a439f98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:39.120960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e374239-f8af-45f2-a02f-4169a439f98e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:39.121459 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e374239-f8af-45f2-a02f-4169a439f98e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:39.121968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e374239-f8af-45f2-a02f-4169a439f98e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:39.138462 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e374239-f8af-45f2-a02f-4169a439f98e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9e374239-f8af-45f2-a02f-4169a439f98e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:39.147212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:39.148119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:39.148885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:39.151948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 73/144] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:16:39.158143 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e80dae-8667-4201-b7cc-7347b3f2643d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:39.326380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e80dae-8667-4201-b7cc-7347b3f2643d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:39.375992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113fe1-4963-48b0-93ca-4ec4a9894a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:39.376697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113fe1-4963-48b0-93ca-4ec4a9894a2f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:39.376985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113fe1-4963-48b0-93ca-4ec4a9894a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:39.377260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113fe1-4963-48b0-93ca-4ec4a9894a2f None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:39.378050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c113fe1-4963-48b0-93ca-4ec4a9894a2f None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:39.378367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c113fe1-4963-48b0-93ca-4ec4a9894a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:39.379749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c113fe1-4963-48b0-93ca-4ec4a9894a2f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:39.395461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 72/145] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:39.400700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e374239-f8af-45f2-a02f-4169a439f98e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:39.518823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:39.556706 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e374239-f8af-45f2-a02f-4169a439f98e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:39.593911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed917e-fa88-4cda-9fe1-075c27533218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:39.594840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed917e-fa88-4cda-9fe1-075c27533218 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:39.595129 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed917e-fa88-4cda-9fe1-075c27533218 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:39.595411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed917e-fa88-4cda-9fe1-075c27533218 None admin] PATH_INFO: `/v3/users/d68b999da6f749cd98aa5b80062e64f3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:39.596706 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed917e-fa88-4cda-9fe1-075c27533218 None admin] RBAC: Authorizing `identity:get_user(user_id=d68b999da6f749cd98aa5b80062e64f3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:39.598204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed917e-fa88-4cda-9fe1-075c27533218 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:39.599880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 74/146] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:39 2022] GET /identity/v3/users/d68b999da6f749cd98aa5b80062e64f3 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:39.605373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113fe1-4963-48b0-93ca-4ec4a9894a2f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:39.674211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:39.737867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113fe1-4963-48b0-93ca-4ec4a9894a2f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:39.785082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87148c5-4560-46b6-8087-4a193fb7f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:39.785758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87148c5-4560-46b6-8087-4a193fb7f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:39.786147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87148c5-4560-46b6-8087-4a193fb7f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:39.786435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87148c5-4560-46b6-8087-4a193fb7f7e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:39.869090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87148c5-4560-46b6-8087-4a193fb7f7e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:39.870602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87148c5-4560-46b6-8087-4a193fb7f7e0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:39.927695 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:39.964886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e2d03-d94e-4558-8c71-3442ffd038bf None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:39.965384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e2d03-d94e-4558-8c71-3442ffd038bf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:39.965470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e2d03-d94e-4558-8c71-3442ffd038bf None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:39.966975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 75/147] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:16:39.972842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbcd04-c505-4224-be0f-d26b66847817 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:39.973129 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbcd04-c505-4224-be0f-d26b66847817 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:39.973411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbcd04-c505-4224-be0f-d26b66847817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:39.995419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d7fbcd04-c505-4224-be0f-d26b66847817 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:39.996613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d7fbcd04-c505-4224-be0f-d26b66847817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:40.018747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 73/148] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:40.025751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87148c5-4560-46b6-8087-4a193fb7f7e0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:40.134038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 76/149] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:40.165050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbcd04-c505-4224-be0f-d26b66847817 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:40.199868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87148c5-4560-46b6-8087-4a193fb7f7e0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:40.253733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8aeb9-78be-4987-be89-92cc659aac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:40.254289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:40.254822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8aeb9-78be-4987-be89-92cc659aac57 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:40.255188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8aeb9-78be-4987-be89-92cc659aac57 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:40.255560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8aeb9-78be-4987-be89-92cc659aac57 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:40.256890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8aeb9-78be-4987-be89-92cc659aac57 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:40.258489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8aeb9-78be-4987-be89-92cc659aac57 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:40.262244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 74/150] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:40 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:40.267279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8aeb9-78be-4987-be89-92cc659aac57 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:40.296964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:40.313545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbcd04-c505-4224-be0f-d26b66847817 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:40.378817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d253077-78ef-4465-a1ca-b0f0a4a42f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:40.380118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d253077-78ef-4465-a1ca-b0f0a4a42f6b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:40.380574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d253077-78ef-4465-a1ca-b0f0a4a42f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:40.380997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d253077-78ef-4465-a1ca-b0f0a4a42f6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:40.401607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d253077-78ef-4465-a1ca-b0f0a4a42f6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6d253077-78ef-4465-a1ca-b0f0a4a42f6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:40.416094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:40.417162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:40.418103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:40.420231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 77/151] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:16:40.426850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d253077-78ef-4465-a1ca-b0f0a4a42f6b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:40.455850 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8aeb9-78be-4987-be89-92cc659aac57 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:40.505085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a43ef-55ce-4224-8284-fde8c6f83a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:40.505764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a43ef-55ce-4224-8284-fde8c6f83a50 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:40.509961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a43ef-55ce-4224-8284-fde8c6f83a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:40.509961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a43ef-55ce-4224-8284-fde8c6f83a50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:40.509961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a43ef-55ce-4224-8284-fde8c6f83a50 None admin] RBAC: Adding query filter params (role.id=d3c75d38368d47d799bf59a0f9c879ea, scope.project.id=2f47c32296eb4caabf57eb7403220ebf, user.id=d68b999da6f749cd98aa5b80062e64f3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:40.509961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a43ef-55ce-4224-8284-fde8c6f83a50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:40.509961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a43ef-55ce-4224-8284-fde8c6f83a50 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:40.525973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 75/152] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:40 2022] GET /identity/v3/role_assignments?user.id=d68b999da6f749cd98aa5b80062e64f3&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:40.604282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d253077-78ef-4465-a1ca-b0f0a4a42f6b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:40.646005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:40.654881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c45960-ea19-4bca-9d84-955f88afe116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:40.656410 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c45960-ea19-4bca-9d84-955f88afe116 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:40.657074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c45960-ea19-4bca-9d84-955f88afe116 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:40.657622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c45960-ea19-4bca-9d84-955f88afe116 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:40.658824 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c45960-ea19-4bca-9d84-955f88afe116 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:40.659142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c45960-ea19-4bca-9d84-955f88afe116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:40.660553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c45960-ea19-4bca-9d84-955f88afe116 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:40.674632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 78/153] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:40.680284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a43ef-55ce-4224-8284-fde8c6f83a50 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:40.803197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:40.825163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a43ef-55ce-4224-8284-fde8c6f83a50 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:40.862548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628dcc62-10b0-4ccd-b452-c0759f9a366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:40.863499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dcc62-10b0-4ccd-b452-c0759f9a366c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:40.863792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dcc62-10b0-4ccd-b452-c0759f9a366c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:40.864074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dcc62-10b0-4ccd-b452-c0759f9a366c None admin] PATH_INFO: `/v3/users/148723ffdfc74b8d87150f004cdfbd01` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:40.865306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628dcc62-10b0-4ccd-b452-c0759f9a366c None admin] RBAC: Authorizing `identity:get_user(user_id=148723ffdfc74b8d87150f004cdfbd01)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:40.866789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628dcc62-10b0-4ccd-b452-c0759f9a366c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:40.868479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 76/154] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:40 2022] GET /identity/v3/users/148723ffdfc74b8d87150f004cdfbd01 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:40.874625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c45960-ea19-4bca-9d84-955f88afe116 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:41.021228 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c45960-ea19-4bca-9d84-955f88afe116 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:41.064253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2ff5d-b0f0-4158-85df-cb7e40aac2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:41.065140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2ff5d-b0f0-4158-85df-cb7e40aac2dc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:41.065423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2ff5d-b0f0-4158-85df-cb7e40aac2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:41.065701 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2ff5d-b0f0-4158-85df-cb7e40aac2dc None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:41.070728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2ff5d-b0f0-4158-85df-cb7e40aac2dc None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:41.072231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2ff5d-b0f0-4158-85df-cb7e40aac2dc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:41.074315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 79/155] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:40 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:41.080242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628dcc62-10b0-4ccd-b452-c0759f9a366c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:41.222428 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628dcc62-10b0-4ccd-b452-c0759f9a366c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:41.272651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf587f-204e-4ef3-8ead-fbfebc39ce05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:41.273899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf587f-204e-4ef3-8ead-fbfebc39ce05 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:41.274441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf587f-204e-4ef3-8ead-fbfebc39ce05 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:41.274935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf587f-204e-4ef3-8ead-fbfebc39ce05 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf/users/148723ffdfc74b8d87150f004cdfbd01/roles/a889423e7cb541dc99e90f4293116033` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:41.277218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf587f-204e-4ef3-8ead-fbfebc39ce05 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f47c32296eb4caabf57eb7403220ebf, user_id=148723ffdfc74b8d87150f004cdfbd01, role_id=a889423e7cb541dc99e90f4293116033)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:41.278934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf587f-204e-4ef3-8ead-fbfebc39ce05 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:41.314060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 77/156] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:16:41 2022] PUT /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf/users/148723ffdfc74b8d87150f004cdfbd01/roles/a889423e7cb541dc99e90f4293116033 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:16:41.839485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e7933-f158-46f9-97e3-f5419f4f09e0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:41.839833 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e7933-f158-46f9-97e3-f5419f4f09e0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:41.840052 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e7933-f158-46f9-97e3-f5419f4f09e0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:41.842364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 80/157] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:41.847721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799a824-ed44-40ae-85b4-8303ca75449c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:41.848326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799a824-ed44-40ae-85b4-8303ca75449c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:41.848326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799a824-ed44-40ae-85b4-8303ca75449c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:41.875012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1799a824-ed44-40ae-85b4-8303ca75449c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:41.876220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1799a824-ed44-40ae-85b4-8303ca75449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:41.880007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:41.967318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 78/158] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:42.002720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e7933-f158-46f9-97e3-f5419f4f09e0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:42.026163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494fd25-b100-4415-84fd-b59604304452 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:42.026500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494fd25-b100-4415-84fd-b59604304452 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:42.026734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494fd25-b100-4415-84fd-b59604304452 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:42.028270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 79/159] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:16:42.034978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300102d-626b-4667-a0bc-f55c8b7f5a37 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:42.035267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300102d-626b-4667-a0bc-f55c8b7f5a37 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:42.035555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300102d-626b-4667-a0bc-f55c8b7f5a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:42.066845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4300102d-626b-4667-a0bc-f55c8b7f5a37 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:42.068046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4300102d-626b-4667-a0bc-f55c8b7f5a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:42.142381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:42.231198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 80/160] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:16:42.232904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17464e-96ab-46ab-978f-753d4bb82c04 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:42.233274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17464e-96ab-46ab-978f-753d4bb82c04 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:42.233560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17464e-96ab-46ab-978f-753d4bb82c04 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:42.235001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 81/161] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:42.235494 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e7933-f158-46f9-97e3-f5419f4f09e0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:42.243213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa475e67-43f7-45b8-982d-81237b0eb5f3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:42.243499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa475e67-43f7-45b8-982d-81237b0eb5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:42.243788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa475e67-43f7-45b8-982d-81237b0eb5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:42.279801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fa475e67-43f7-45b8-982d-81237b0eb5f3 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:42.281007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fa475e67-43f7-45b8-982d-81237b0eb5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:42.333116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5efa8-4d49-4b8f-9d49-cc4abd32bd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:42.333964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5efa8-4d49-4b8f-9d49-cc4abd32bd9f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:42.334263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5efa8-4d49-4b8f-9d49-cc4abd32bd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:42.334539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5efa8-4d49-4b8f-9d49-cc4abd32bd9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:42.375991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:42.382552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5efa8-4d49-4b8f-9d49-cc4abd32bd9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:42.383937 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 82/162] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:42.384567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa475e67-43f7-45b8-982d-81237b0eb5f3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ede5efa8-4d49-4b8f-9d49-cc4abd32bd9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:42.391790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:42.392704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:42.393876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 81/163] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:16:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:42.410956 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5efa8-4d49-4b8f-9d49-cc4abd32bd9f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:42.512731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa475e67-43f7-45b8-982d-81237b0eb5f3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:42.574650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089688c4-be7f-4337-b040-f1d8bb9677b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:42.576642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089688c4-be7f-4337-b040-f1d8bb9677b8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:42.577127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089688c4-be7f-4337-b040-f1d8bb9677b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:42.582534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089688c4-be7f-4337-b040-f1d8bb9677b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:42.602740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089688c4-be7f-4337-b040-f1d8bb9677b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-089688c4-be7f-4337-b040-f1d8bb9677b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:42.611118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:42.611958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:42.612725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:42.622943 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 83/164] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:42.624283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089688c4-be7f-4337-b040-f1d8bb9677b8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:42.655856 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5efa8-4d49-4b8f-9d49-cc4abd32bd9f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:42.685194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:42.723565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b1dd0-7cb0-4ab7-be83-bca827378692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:42.724255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b1dd0-7cb0-4ab7-be83-bca827378692 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:42.724538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b1dd0-7cb0-4ab7-be83-bca827378692 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:42.724815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b1dd0-7cb0-4ab7-be83-bca827378692 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:42.725585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b1dd0-7cb0-4ab7-be83-bca827378692 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:42.725896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b1dd0-7cb0-4ab7-be83-bca827378692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:42.727306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b1dd0-7cb0-4ab7-be83-bca827378692 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:42.740427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 82/165] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:16:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:42.741001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:42.741428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b1dd0-7cb0-4ab7-be83-bca827378692 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:42.824257 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089688c4-be7f-4337-b040-f1d8bb9677b8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:42.910468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924e7b9-f628-4e69-b099-0849ea2b68c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:42.912236 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e7b9-f628-4e69-b099-0849ea2b68c1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:42.914092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e7b9-f628-4e69-b099-0849ea2b68c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:42.914578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e7b9-f628-4e69-b099-0849ea2b68c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:42.936501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924e7b9-f628-4e69-b099-0849ea2b68c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:42.940723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b1dd0-7cb0-4ab7-be83-bca827378692 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0924e7b9-f628-4e69-b099-0849ea2b68c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:42.945000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:42.945851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:42.946667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:42.949043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 84/166] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:16:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:42.955072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924e7b9-f628-4e69-b099-0849ea2b68c1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:43.005814 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3083e7-2d79-488d-b477-5be4e1ed8782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:43.006706 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3083e7-2d79-488d-b477-5be4e1ed8782 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:43.006836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3083e7-2d79-488d-b477-5be4e1ed8782 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:43.007081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3083e7-2d79-488d-b477-5be4e1ed8782 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:43.007918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3083e7-2d79-488d-b477-5be4e1ed8782 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:43.008227 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3083e7-2d79-488d-b477-5be4e1ed8782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:43.009615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3083e7-2d79-488d-b477-5be4e1ed8782 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:43.025173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 83/167] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:43.028197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3083e7-2d79-488d-b477-5be4e1ed8782 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:43.091674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924e7b9-f628-4e69-b099-0849ea2b68c1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:43.113100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:43.136615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:43.138866 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ecced-e044-4273-a89f-9d7997483c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:43.140133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ecced-e044-4273-a89f-9d7997483c1f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:43.140601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ecced-e044-4273-a89f-9d7997483c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:43.141019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ecced-e044-4273-a89f-9d7997483c1f None admin] PATH_INFO: `/v3/users/d68b999da6f749cd98aa5b80062e64f3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:43.142600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ecced-e044-4273-a89f-9d7997483c1f None admin] RBAC: Authorizing `identity:get_user(user_id=d68b999da6f749cd98aa5b80062e64f3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:43.145083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ecced-e044-4273-a89f-9d7997483c1f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:43.147108 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 85/168] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:42 2022] GET /identity/v3/users/d68b999da6f749cd98aa5b80062e64f3 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:43.148437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541dc067-54d2-40b8-96c5-4b370f6d3dd0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:43.148755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541dc067-54d2-40b8-96c5-4b370f6d3dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:43.149018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541dc067-54d2-40b8-96c5-4b370f6d3dd0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:43.150379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 86/169] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:43.151153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541dc067-54d2-40b8-96c5-4b370f6d3dd0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:43.232794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3083e7-2d79-488d-b477-5be4e1ed8782 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:43.289394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd41f76-3238-46e7-864b-70330710b962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:43.290448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41f76-3238-46e7-864b-70330710b962 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:43.290926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41f76-3238-46e7-864b-70330710b962 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:43.291431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41f76-3238-46e7-864b-70330710b962 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:43.292394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd41f76-3238-46e7-864b-70330710b962 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:43.292955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd41f76-3238-46e7-864b-70330710b962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:43.296463 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd41f76-3238-46e7-864b-70330710b962 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:43.312909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 84/170] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:43.315515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea38e20-9e53-4799-9de0-e2040a9d39fd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:43.315814 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea38e20-9e53-4799-9de0-e2040a9d39fd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:43.316101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea38e20-9e53-4799-9de0-e2040a9d39fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:43.330330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541dc067-54d2-40b8-96c5-4b370f6d3dd0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:43.347085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dea38e20-9e53-4799-9de0-e2040a9d39fd None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:43.348312 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dea38e20-9e53-4799-9de0-e2040a9d39fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:43.374591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139b06e-4c1c-45a6-80ee-d684e155cf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:43.376740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139b06e-4c1c-45a6-80ee-d684e155cf12 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:43.377217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139b06e-4c1c-45a6-80ee-d684e155cf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:43.378267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139b06e-4c1c-45a6-80ee-d684e155cf12 None admin] PATH_INFO: `/v3/users/d68b999da6f749cd98aa5b80062e64f3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:43.380055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139b06e-4c1c-45a6-80ee-d684e155cf12 None admin] RBAC: Authorizing `identity:get_user(user_id=d68b999da6f749cd98aa5b80062e64f3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:43.383212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139b06e-4c1c-45a6-80ee-d684e155cf12 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:43.388905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 87/171] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:43 2022] GET /identity/v3/users/d68b999da6f749cd98aa5b80062e64f3 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:43.390062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139b06e-4c1c-45a6-80ee-d684e155cf12 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:43.456191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:43.504675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139b06e-4c1c-45a6-80ee-d684e155cf12 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:43.527318 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 85/172] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:43.528718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea38e20-9e53-4799-9de0-e2040a9d39fd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:43.555626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd82f5-44e2-4625-ae11-15d67c6a2ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:43.556301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd82f5-44e2-4625-ae11-15d67c6a2ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:43.556584 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd82f5-44e2-4625-ae11-15d67c6a2ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:43.556858 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd82f5-44e2-4625-ae11-15d67c6a2ca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:43.598402 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:43.676251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea38e20-9e53-4799-9de0-e2040a9d39fd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:43.693356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd82f5-44e2-4625-ae11-15d67c6a2ca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:43.694877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd82f5-44e2-4625-ae11-15d67c6a2ca6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:43.747822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36647d-ce5c-4c69-bf2d-3bdcbdd0c2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:43.748718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36647d-ce5c-4c69-bf2d-3bdcbdd0c2eb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:43.749000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36647d-ce5c-4c69-bf2d-3bdcbdd0c2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:43.749274 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36647d-ce5c-4c69-bf2d-3bdcbdd0c2eb None admin] PATH_INFO: `/v3/users/a1033721c1da485882816b21d7d01a42` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:43.753682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36647d-ce5c-4c69-bf2d-3bdcbdd0c2eb None admin] RBAC: Authorizing `identity:get_user(user_id=a1033721c1da485882816b21d7d01a42)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:43.755243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36647d-ce5c-4c69-bf2d-3bdcbdd0c2eb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:43.756869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 86/173] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:43 2022] GET /identity/v3/users/a1033721c1da485882816b21d7d01a42 => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:43.757638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36647d-ce5c-4c69-bf2d-3bdcbdd0c2eb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:43.807717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:43.891909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 88/174] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:43.892790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd82f5-44e2-4625-ae11-15d67c6a2ca6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:43.947738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36647d-ce5c-4c69-bf2d-3bdcbdd0c2eb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:43.949309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:43.982835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b56bbb-4def-4634-ae29-b4a7703bc83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:43.983487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b56bbb-4def-4634-ae29-b4a7703bc83b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:43.983947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b56bbb-4def-4634-ae29-b4a7703bc83b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:43.984223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b56bbb-4def-4634-ae29-b4a7703bc83b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.016858 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd82f5-44e2-4625-ae11-15d67c6a2ca6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:44.058188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b56bbb-4def-4634-ae29-b4a7703bc83b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:44.060549 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b56bbb-4def-4634-ae29-b4a7703bc83b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:44.062383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e727d-dafc-4161-886e-db59cba91af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:44.063187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e727d-dafc-4161-886e-db59cba91af6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.063466 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e727d-dafc-4161-886e-db59cba91af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.063740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e727d-dafc-4161-886e-db59cba91af6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.074197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e727d-dafc-4161-886e-db59cba91af6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a79e727d-dafc-4161-886e-db59cba91af6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:44.081929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:44.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 89/175] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:16:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:44.088794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e727d-dafc-4161-886e-db59cba91af6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:44.105293 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:44.176463 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 87/176] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:44.177312 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b56bbb-4def-4634-ae29-b4a7703bc83b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:44.179576 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:44.194217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e727d-dafc-4161-886e-db59cba91af6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:44.232106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fac6d7-4c36-4ccc-b64b-e2c0f3f00075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:44.232480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fac6d7-4c36-4ccc-b64b-e2c0f3f00075 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.232900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fac6d7-4c36-4ccc-b64b-e2c0f3f00075 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.233185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fac6d7-4c36-4ccc-b64b-e2c0f3f00075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.278420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b56bbb-4def-4634-ae29-b4a7703bc83b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:44.304053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fac6d7-4c36-4ccc-b64b-e2c0f3f00075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:44.305518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fac6d7-4c36-4ccc-b64b-e2c0f3f00075 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:44.312001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f342b-a0a3-48f5-abdf-a7cf14524f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:44.312869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f342b-a0a3-48f5-abdf-a7cf14524f45 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.313155 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f342b-a0a3-48f5-abdf-a7cf14524f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.313431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f342b-a0a3-48f5-abdf-a7cf14524f45 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.314641 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f342b-a0a3-48f5-abdf-a7cf14524f45 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:44.316070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f342b-a0a3-48f5-abdf-a7cf14524f45 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:44.317766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 88/177] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:44 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:44.319050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dec59-f449-456e-8417-89ca1e259a40 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.319341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dec59-f449-456e-8417-89ca1e259a40 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.319624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dec59-f449-456e-8417-89ca1e259a40 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.320832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 89/178] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:44.322204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ae073-9689-4728-812c-eb9352950f02 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.322613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ae073-9689-4728-812c-eb9352950f02 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.323066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ae073-9689-4728-812c-eb9352950f02 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.324292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 90/179] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:44.325306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ae073-9689-4728-812c-eb9352950f02 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:44.326399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:44.398429 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:44.410105 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 90/180] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:44.411239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fac6d7-4c36-4ccc-b64b-e2c0f3f00075 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:44.434877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ae073-9689-4728-812c-eb9352950f02 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:44.468422 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2ce5d-4e1c-4570-b25c-e590c4c402fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:44.469088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ce5d-4e1c-4570-b25c-e590c4c402fd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.469378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ce5d-4e1c-4570-b25c-e590c4c402fd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.469654 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ce5d-4e1c-4570-b25c-e590c4c402fd None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.470444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2ce5d-4e1c-4570-b25c-e590c4c402fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:44.470755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2ce5d-4e1c-4570-b25c-e590c4c402fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:44.472141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2ce5d-4e1c-4570-b25c-e590c4c402fd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:44.480227 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 91/181] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:44.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e4b26-762b-476f-8743-875341379390 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.482242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e4b26-762b-476f-8743-875341379390 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.482524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e4b26-762b-476f-8743-875341379390 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.483771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 92/182] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:44.484469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e4b26-762b-476f-8743-875341379390 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:44.516980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fac6d7-4c36-4ccc-b64b-e2c0f3f00075 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:44.533562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:44.552651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d7b93-1779-4396-9151-9af91eca0221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:44.553514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7b93-1779-4396-9151-9af91eca0221 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.553802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7b93-1779-4396-9151-9af91eca0221 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.554113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7b93-1779-4396-9151-9af91eca0221 None admin] PATH_INFO: `/v3/projects/5278726ed9db4870bb01682315cbb3e6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.555299 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d7b93-1779-4396-9151-9af91eca0221 None admin] RBAC: Authorizing `identity:get_project(project_id=5278726ed9db4870bb01682315cbb3e6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:44.556761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d7b93-1779-4396-9151-9af91eca0221 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:44.558655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 91/183] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:44 2022] GET /identity/v3/projects/5278726ed9db4870bb01682315cbb3e6 => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:44.559713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8333c0a-8a48-4a48-90bc-9a3b5c7c1885 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.560001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8333c0a-8a48-4a48-90bc-9a3b5c7c1885 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.560278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8333c0a-8a48-4a48-90bc-9a3b5c7c1885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.579232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a8333c0a-8a48-4a48-90bc-9a3b5c7c1885 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:44.580416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a8333c0a-8a48-4a48-90bc-9a3b5c7c1885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:44.587224 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e4b26-762b-476f-8743-875341379390 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:44.615960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:44.621088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f2ebf-e6b3-47fb-ada3-9b2edd49fdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:44.621905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f2ebf-e6b3-47fb-ada3-9b2edd49fdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.622368 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f2ebf-e6b3-47fb-ada3-9b2edd49fdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.622654 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f2ebf-e6b3-47fb-ada3-9b2edd49fdf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.623499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f2ebf-e6b3-47fb-ada3-9b2edd49fdf3 None admin] RBAC: Adding query filter params (role.id=d534ab3d0a904f0babeb90d2679d1b86, scope.project.id=2f47c32296eb4caabf57eb7403220ebf, user.id=d68b999da6f749cd98aa5b80062e64f3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:44.623817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f2ebf-e6b3-47fb-ada3-9b2edd49fdf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:44.625283 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f2ebf-e6b3-47fb-ada3-9b2edd49fdf3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:44.631163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 93/184] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:44 2022] GET /identity/v3/role_assignments?user.id=d68b999da6f749cd98aa5b80062e64f3&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=d534ab3d0a904f0babeb90d2679d1b86 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:44.632327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3216702-746a-48c5-a881-afdf87b6539c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.632616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3216702-746a-48c5-a881-afdf87b6539c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.632897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3216702-746a-48c5-a881-afdf87b6539c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.647618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 92/185] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:44.647618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8333c0a-8a48-4a48-90bc-9a3b5c7c1885 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:44.665107 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f3216702-746a-48c5-a881-afdf87b6539c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:44.667244 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f3216702-746a-48c5-a881-afdf87b6539c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:44.754657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 94/186] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:44.754657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3216702-746a-48c5-a881-afdf87b6539c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:44.771798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8333c0a-8a48-4a48-90bc-9a3b5c7c1885 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:44.792826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:44.820643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80719baf-2d3c-415f-8caa-907d4b4ab0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:44.821533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80719baf-2d3c-415f-8caa-907d4b4ab0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.821823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80719baf-2d3c-415f-8caa-907d4b4ab0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.822139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80719baf-2d3c-415f-8caa-907d4b4ab0c3 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.823345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80719baf-2d3c-415f-8caa-907d4b4ab0c3 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:44.825541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80719baf-2d3c-415f-8caa-907d4b4ab0c3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:44.828478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 93/187] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:44 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:44.828478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf310-b53f-4b9d-8cd8-9d2af1dbd86d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.828778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf310-b53f-4b9d-8cd8-9d2af1dbd86d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.830154 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf310-b53f-4b9d-8cd8-9d2af1dbd86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.847303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-74fcf310-b53f-4b9d-8cd8-9d2af1dbd86d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:44.848495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-74fcf310-b53f-4b9d-8cd8-9d2af1dbd86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:44.875803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3216702-746a-48c5-a881-afdf87b6539c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:44.901541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:44.911647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a04a6-6da6-4cfc-8e33-d4e694e91805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:44.912782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a04a6-6da6-4cfc-8e33-d4e694e91805 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:44.913130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a04a6-6da6-4cfc-8e33-d4e694e91805 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:44.913418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a04a6-6da6-4cfc-8e33-d4e694e91805 None admin] PATH_INFO: `/v3/users/3875dea0920542f28b6e5f3ba2586b74` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:44.918666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 94/188] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:44.922881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcf310-b53f-4b9d-8cd8-9d2af1dbd86d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:44.927332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a04a6-6da6-4cfc-8e33-d4e694e91805 None admin] RBAC: Authorizing `identity:get_user(user_id=3875dea0920542f28b6e5f3ba2586b74)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:44.928777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a04a6-6da6-4cfc-8e33-d4e694e91805 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:44.930643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 95/189] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:44 2022] GET /identity/v3/users/3875dea0920542f28b6e5f3ba2586b74 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:44.935919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a04a6-6da6-4cfc-8e33-d4e694e91805 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:45.022156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcf310-b53f-4b9d-8cd8-9d2af1dbd86d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:45.034924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:45.035048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a04a6-6da6-4cfc-8e33-d4e694e91805 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:45.057253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e7453-19b9-46ca-a818-65fc0665f35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:45.057925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e7453-19b9-46ca-a818-65fc0665f35d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:45.058239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e7453-19b9-46ca-a818-65fc0665f35d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:45.058518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e7453-19b9-46ca-a818-65fc0665f35d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:45.059915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e7453-19b9-46ca-a818-65fc0665f35d None admin] RBAC: Adding query filter params (role.id=d3c75d38368d47d799bf59a0f9c879ea, scope.project.id=5278726ed9db4870bb01682315cbb3e6, user.id=d68b999da6f749cd98aa5b80062e64f3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:45.060226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e7453-19b9-46ca-a818-65fc0665f35d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:45.061673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e7453-19b9-46ca-a818-65fc0665f35d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:45.067657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 95/190] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:44 2022] GET /identity/v3/role_assignments?user.id=d68b999da6f749cd98aa5b80062e64f3&scope.project.id=5278726ed9db4870bb01682315cbb3e6&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:45.075698 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48258c-8a82-4538-8915-f7a496ffd09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:45.076140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e7453-19b9-46ca-a818-65fc0665f35d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:45.076657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48258c-8a82-4538-8915-f7a496ffd09b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:45.076948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48258c-8a82-4538-8915-f7a496ffd09b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:45.077223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48258c-8a82-4538-8915-f7a496ffd09b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:45.091662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d48258c-8a82-4538-8915-f7a496ffd09b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9d48258c-8a82-4538-8915-f7a496ffd09b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:45.102408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:45.103393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:45.104177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 96/191] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:16:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:45.105617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48258c-8a82-4538-8915-f7a496ffd09b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:45.159683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:45.205203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e7453-19b9-46ca-a818-65fc0665f35d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:45.219340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48258c-8a82-4538-8915-f7a496ffd09b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:45.250918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb08380-6237-4dd3-a7c3-527069129f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:45.251727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb08380-6237-4dd3-a7c3-527069129f8d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:45.252011 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb08380-6237-4dd3-a7c3-527069129f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:45.252289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb08380-6237-4dd3-a7c3-527069129f8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:45.253469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb08380-6237-4dd3-a7c3-527069129f8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:45.254969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb08380-6237-4dd3-a7c3-527069129f8d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:45.256630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 96/192] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:45.257385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb08380-6237-4dd3-a7c3-527069129f8d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:45.261064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c76aae-e415-44e1-8b9f-5f2914d42708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:45.261733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c76aae-e415-44e1-8b9f-5f2914d42708 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:45.262896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c76aae-e415-44e1-8b9f-5f2914d42708 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:45.262896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c76aae-e415-44e1-8b9f-5f2914d42708 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:45.263818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c76aae-e415-44e1-8b9f-5f2914d42708 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=2f47c32296eb4caabf57eb7403220ebf, user.id=a1033721c1da485882816b21d7d01a42) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:45.264139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c76aae-e415-44e1-8b9f-5f2914d42708 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:45.266710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c76aae-e415-44e1-8b9f-5f2914d42708 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:45.273699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 97/193] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:45 2022] GET /identity/v3/role_assignments?user.id=a1033721c1da485882816b21d7d01a42&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:45.274969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c76aae-e415-44e1-8b9f-5f2914d42708 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:45.327462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:45.388361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb08380-6237-4dd3-a7c3-527069129f8d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:45.402242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c76aae-e415-44e1-8b9f-5f2914d42708 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:45.413536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:45.428503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bfcf8-253e-4a8a-a7c1-bb9c772e9f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:45.429104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bfcf8-253e-4a8a-a7c1-bb9c772e9f4b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:45.429390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bfcf8-253e-4a8a-a7c1-bb9c772e9f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:45.429670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bfcf8-253e-4a8a-a7c1-bb9c772e9f4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:45.449727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a8f38-6ecc-4db5-a2a1-202677cfee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:45.451379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a8f38-6ecc-4db5-a2a1-202677cfee30 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:45.451379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a8f38-6ecc-4db5-a2a1-202677cfee30 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:45.451379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a8f38-6ecc-4db5-a2a1-202677cfee30 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:45.466535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a8f38-6ecc-4db5-a2a1-202677cfee30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4d0a8f38-6ecc-4db5-a2a1-202677cfee30 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:45.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:45.475617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:45.476406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 98/194] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:45.477067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a8f38-6ecc-4db5-a2a1-202677cfee30 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:45.511927 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bfcf8-253e-4a8a-a7c1-bb9c772e9f4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:45.513412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bfcf8-253e-4a8a-a7c1-bb9c772e9f4b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:45.558543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:45.589376 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a8f38-6ecc-4db5-a2a1-202677cfee30 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:45.615772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 97/195] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:45.615772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bfcf8-253e-4a8a-a7c1-bb9c772e9f4b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:45.627512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a6d9c-9a5a-42ab-b603-abe3bf7bcede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:45.628485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6d9c-9a5a-42ab-b603-abe3bf7bcede None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:45.628767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6d9c-9a5a-42ab-b603-abe3bf7bcede None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:45.629044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6d9c-9a5a-42ab-b603-abe3bf7bcede None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:45.629823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a6d9c-9a5a-42ab-b603-abe3bf7bcede None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:45.630160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a6d9c-9a5a-42ab-b603-abe3bf7bcede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:45.631556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a6d9c-9a5a-42ab-b603-abe3bf7bcede None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:45.639582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 99/196] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:16:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:45.640358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a6d9c-9a5a-42ab-b603-abe3bf7bcede None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:45.674584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:45.718193 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bfcf8-253e-4a8a-a7c1-bb9c772e9f4b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:45.745026 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a6d9c-9a5a-42ab-b603-abe3bf7bcede None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:45.751316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031daff-3dba-43d7-bc49-affc45f21c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:45.751984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031daff-3dba-43d7-bc49-affc45f21c01 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:45.752261 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031daff-3dba-43d7-bc49-affc45f21c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:45.752535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031daff-3dba-43d7-bc49-affc45f21c01 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:45.753235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031daff-3dba-43d7-bc49-affc45f21c01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:45.754689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031daff-3dba-43d7-bc49-affc45f21c01 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:45.770579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:45.775428 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 98/197] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:16:45 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:45.776377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031daff-3dba-43d7-bc49-affc45f21c01 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:45.780388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961af8d4-9ac7-457c-b2a6-85e8b83038f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:45.781566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961af8d4-9ac7-457c-b2a6-85e8b83038f0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:45.781883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961af8d4-9ac7-457c-b2a6-85e8b83038f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:45.782306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961af8d4-9ac7-457c-b2a6-85e8b83038f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:45.783344 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961af8d4-9ac7-457c-b2a6-85e8b83038f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:45.783661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961af8d4-9ac7-457c-b2a6-85e8b83038f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:45.785090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961af8d4-9ac7-457c-b2a6-85e8b83038f0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:45.795989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 100/198] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:45.795989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961af8d4-9ac7-457c-b2a6-85e8b83038f0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:45.899635 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:45.909500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031daff-3dba-43d7-bc49-affc45f21c01 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:45.909923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961af8d4-9ac7-457c-b2a6-85e8b83038f0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:45.951274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcefaeb-9a4f-4eed-9d88-d70fce417f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:45.952424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985771-9b21-438c-b8a7-25336ff39764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:45.952590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcefaeb-9a4f-4eed-9d88-d70fce417f47 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:45.952786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcefaeb-9a4f-4eed-9d88-d70fce417f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:45.952987 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcefaeb-9a4f-4eed-9d88-d70fce417f47 None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:45.953311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985771-9b21-438c-b8a7-25336ff39764 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:45.953713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985771-9b21-438c-b8a7-25336ff39764 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:45.954041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985771-9b21-438c-b8a7-25336ff39764 None admin] PATH_INFO: `/v3/users/3875dea0920542f28b6e5f3ba2586b74` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:45.954231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcefaeb-9a4f-4eed-9d88-d70fce417f47 None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:45.955261 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3985771-9b21-438c-b8a7-25336ff39764 None admin] RBAC: Authorizing `identity:get_user(user_id=3875dea0920542f28b6e5f3ba2586b74)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:45.955680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcefaeb-9a4f-4eed-9d88-d70fce417f47 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:45.956710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3985771-9b21-438c-b8a7-25336ff39764 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:45.957432 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 99/199] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:45 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:45.958218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcefaeb-9a4f-4eed-9d88-d70fce417f47 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:45.958381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 101/200] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:45 2022] GET /identity/v3/users/3875dea0920542f28b6e5f3ba2586b74 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:45.963798 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985771-9b21-438c-b8a7-25336ff39764 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:46.057365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:46.088038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985771-9b21-438c-b8a7-25336ff39764 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:46.098467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcefaeb-9a4f-4eed-9d88-d70fce417f47 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:46.128968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bc80c-0842-489f-aeff-35f96ccf4397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:46.129539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bc80c-0842-489f-aeff-35f96ccf4397 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:46.129830 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bc80c-0842-489f-aeff-35f96ccf4397 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:46.130143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bc80c-0842-489f-aeff-35f96ccf4397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:46.130999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bc80c-0842-489f-aeff-35f96ccf4397 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=860106fdc2dc43498b8d25ed76d618e8, user.id=3875dea0920542f28b6e5f3ba2586b74) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:46.131315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bc80c-0842-489f-aeff-35f96ccf4397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:46.132772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bc80c-0842-489f-aeff-35f96ccf4397 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:46.146006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 102/201] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:45 2022] GET /identity/v3/role_assignments?user.id=3875dea0920542f28b6e5f3ba2586b74&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:46.146006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bc80c-0842-489f-aeff-35f96ccf4397 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:46.151636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4a325-1970-48dd-815b-788437a92d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:46.152900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4a325-1970-48dd-815b-788437a92d7e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:46.153372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4a325-1970-48dd-815b-788437a92d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:46.153847 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4a325-1970-48dd-815b-788437a92d7e None admin] PATH_INFO: `/v3/users/148723ffdfc74b8d87150f004cdfbd01` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:46.155458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4a325-1970-48dd-815b-788437a92d7e None admin] RBAC: Authorizing `identity:get_user(user_id=148723ffdfc74b8d87150f004cdfbd01)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:46.158040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4a325-1970-48dd-815b-788437a92d7e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:46.160140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 100/202] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:45 2022] GET /identity/v3/users/148723ffdfc74b8d87150f004cdfbd01 => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:46.180325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4a325-1970-48dd-815b-788437a92d7e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:46.196599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:46.302328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4a325-1970-48dd-815b-788437a92d7e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:46.333368 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bc80c-0842-489f-aeff-35f96ccf4397 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:46.354644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fb7cf-e3d7-4f63-9d93-8926e1ee5bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:46.356230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fb7cf-e3d7-4f63-9d93-8926e1ee5bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:46.356825 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fb7cf-e3d7-4f63-9d93-8926e1ee5bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:46.358156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fb7cf-e3d7-4f63-9d93-8926e1ee5bc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:46.367500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:46.395122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd1d06-6efe-47b4-a9d2-866ac5c34847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:46.395793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd1d06-6efe-47b4-a9d2-866ac5c34847 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:46.396132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd1d06-6efe-47b4-a9d2-866ac5c34847 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:46.396406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd1d06-6efe-47b4-a9d2-866ac5c34847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:46.459775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fb7cf-e3d7-4f63-9d93-8926e1ee5bc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:46.461558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fb7cf-e3d7-4f63-9d93-8926e1ee5bc1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:46.479058 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd1d06-6efe-47b4-a9d2-866ac5c34847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:46.480586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd1d06-6efe-47b4-a9d2-866ac5c34847 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:46.514414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:46.575715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 101/203] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 06:16:46.576946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fb7cf-e3d7-4f63-9d93-8926e1ee5bc1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:46.634036 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 103/204] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:46.638185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:46.639874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd1d06-6efe-47b4-a9d2-866ac5c34847 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:46.745412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd1d06-6efe-47b4-a9d2-866ac5c34847 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:46.782982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d9112-ec47-4b09-ac85-e9058c56916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:46.784409 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d9112-ec47-4b09-ac85-e9058c56916e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:46.784889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d9112-ec47-4b09-ac85-e9058c56916e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:46.785359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d9112-ec47-4b09-ac85-e9058c56916e None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:46.787820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d9112-ec47-4b09-ac85-e9058c56916e None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:46.789673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d9112-ec47-4b09-ac85-e9058c56916e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:46.791979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 104/205] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:46 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:46.792528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:46.799656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d9112-ec47-4b09-ac85-e9058c56916e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:46.908310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d9112-ec47-4b09-ac85-e9058c56916e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:46.908533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fb7cf-e3d7-4f63-9d93-8926e1ee5bc1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:46.947719 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb755a3-e347-4afd-8852-c34879f06489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:46.948940 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb755a3-e347-4afd-8852-c34879f06489 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:46.949485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb755a3-e347-4afd-8852-c34879f06489 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:46.949997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb755a3-e347-4afd-8852-c34879f06489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:46.951132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb755a3-e347-4afd-8852-c34879f06489 None admin] RBAC: Adding query filter params (role.id=d534ab3d0a904f0babeb90d2679d1b86, scope.project.id=860106fdc2dc43498b8d25ed76d618e8, user.id=3875dea0920542f28b6e5f3ba2586b74) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:46.951666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb755a3-e347-4afd-8852-c34879f06489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:46.953294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb755a3-e347-4afd-8852-c34879f06489 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:46.969278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 105/206] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:46 2022] GET /identity/v3/role_assignments?user.id=3875dea0920542f28b6e5f3ba2586b74&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=d534ab3d0a904f0babeb90d2679d1b86 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:46.987700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779ec73-7c23-4d65-b5a5-dba44f50cc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:46.988607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779ec73-7c23-4d65-b5a5-dba44f50cc0f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:46.988892 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779ec73-7c23-4d65-b5a5-dba44f50cc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:46.989170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779ec73-7c23-4d65-b5a5-dba44f50cc0f None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:46.990525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779ec73-7c23-4d65-b5a5-dba44f50cc0f None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:46.992191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779ec73-7c23-4d65-b5a5-dba44f50cc0f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:46.994186 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 102/207] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:46 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:47.000598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb755a3-e347-4afd-8852-c34879f06489 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:47.076714 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:47.155566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb755a3-e347-4afd-8852-c34879f06489 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:47.203602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25776862-3f2d-4add-81b2-0f50a28fe9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:47.204650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25776862-3f2d-4add-81b2-0f50a28fe9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:47.205129 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25776862-3f2d-4add-81b2-0f50a28fe9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:47.205632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25776862-3f2d-4add-81b2-0f50a28fe9b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:47.206760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25776862-3f2d-4add-81b2-0f50a28fe9b5 None admin] RBAC: Adding query filter params (role.id=a889423e7cb541dc99e90f4293116033, scope.project.id=2f47c32296eb4caabf57eb7403220ebf, user.id=148723ffdfc74b8d87150f004cdfbd01) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:47.207293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25776862-3f2d-4add-81b2-0f50a28fe9b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:47.209013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25776862-3f2d-4add-81b2-0f50a28fe9b5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:47.224575 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 106/208] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:46 2022] GET /identity/v3/role_assignments?user.id=148723ffdfc74b8d87150f004cdfbd01&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=a889423e7cb541dc99e90f4293116033 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:48.195342 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588c915-ff8b-4957-af17-31caaa77b06e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:48.195876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588c915-ff8b-4957-af17-31caaa77b06e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:48.195952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588c915-ff8b-4957-af17-31caaa77b06e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:48.197443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 103/209] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:48.203102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963af2a-383a-408f-8cb3-e3efef8e1e02 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:48.203986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963af2a-383a-408f-8cb3-e3efef8e1e02 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:48.204481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963af2a-383a-408f-8cb3-e3efef8e1e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:48.221994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7963af2a-383a-408f-8cb3-e3efef8e1e02 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:48.223463 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7963af2a-383a-408f-8cb3-e3efef8e1e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:48.243299 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353d6ea-aec8-4e5b-82af-547e9390ab68 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:48.243890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353d6ea-aec8-4e5b-82af-547e9390ab68 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:48.244609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353d6ea-aec8-4e5b-82af-547e9390ab68 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:48.246305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 104/210] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:16:48.253883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a025358-b27b-40f6-9f5f-d1257713e867 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:48.254441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a025358-b27b-40f6-9f5f-d1257713e867 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:48.254926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a025358-b27b-40f6-9f5f-d1257713e867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:48.277370 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0a025358-b27b-40f6-9f5f-d1257713e867 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:48.278890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0a025358-b27b-40f6-9f5f-d1257713e867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:48.301728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 107/211] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:48.326509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963af2a-383a-408f-8cb3-e3efef8e1e02 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:48.358309 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 105/212] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:16:48.376270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a025358-b27b-40f6-9f5f-d1257713e867 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:48.382548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:48.414363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:48.433341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963af2a-383a-408f-8cb3-e3efef8e1e02 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:48.469883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9ebeb-f3dc-4e1b-a80d-4fd1c9faffd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:48.470952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9ebeb-f3dc-4e1b-a80d-4fd1c9faffd2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:48.471407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9ebeb-f3dc-4e1b-a80d-4fd1c9faffd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:48.471830 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9ebeb-f3dc-4e1b-a80d-4fd1c9faffd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:48.473157 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9ebeb-f3dc-4e1b-a80d-4fd1c9faffd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:48.474807 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9ebeb-f3dc-4e1b-a80d-4fd1c9faffd2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:48.476575 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 108/213] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:48.482232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9ebeb-f3dc-4e1b-a80d-4fd1c9faffd2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:48.487449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a025358-b27b-40f6-9f5f-d1257713e867 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:48.523570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e45d3-6596-419f-b548-1b7dd740e741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:48.524633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e45d3-6596-419f-b548-1b7dd740e741 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:48.525131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e45d3-6596-419f-b548-1b7dd740e741 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:48.525619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e45d3-6596-419f-b548-1b7dd740e741 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:48.538554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e45d3-6596-419f-b548-1b7dd740e741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0a7e45d3-6596-419f-b548-1b7dd740e741 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:48.547391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:48.548280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:48.549050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 106/214] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:16:48.556113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e45d3-6596-419f-b548-1b7dd740e741 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:48.596881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9ebeb-f3dc-4e1b-a80d-4fd1c9faffd2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:48.616032 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:48.637580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc5361-643b-41e7-8125-d726e3537c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:48.638514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc5361-643b-41e7-8125-d726e3537c81 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:48.638962 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc5361-643b-41e7-8125-d726e3537c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:48.639379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc5361-643b-41e7-8125-d726e3537c81 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:48.640636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc5361-643b-41e7-8125-d726e3537c81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:48.642882 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc5361-643b-41e7-8125-d726e3537c81 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:48.662224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:48.665792 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 109/215] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:16:48 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:48.667271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bf9eb-42c6-4835-b563-220487982cc3 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:48.667568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bf9eb-42c6-4835-b563-220487982cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:48.667846 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bf9eb-42c6-4835-b563-220487982cc3 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:48.669133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 110/216] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:48.671609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e45d3-6596-419f-b548-1b7dd740e741 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:48.675029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3cd6c-2114-4cd5-950d-f6b3112b59b1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:48.675343 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3cd6c-2114-4cd5-950d-f6b3112b59b1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:48.675603 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3cd6c-2114-4cd5-950d-f6b3112b59b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:48.699535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-83e3cd6c-2114-4cd5-950d-f6b3112b59b1 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:48.701394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-83e3cd6c-2114-4cd5-950d-f6b3112b59b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:48.726104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328b9fa-ea86-497f-b138-f6b5ed9611cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:48.727780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328b9fa-ea86-497f-b138-f6b5ed9611cf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:48.728250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328b9fa-ea86-497f-b138-f6b5ed9611cf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:48.728707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328b9fa-ea86-497f-b138-f6b5ed9611cf None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:48.729764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328b9fa-ea86-497f-b138-f6b5ed9611cf None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:48.730678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328b9fa-ea86-497f-b138-f6b5ed9611cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:48.733130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328b9fa-ea86-497f-b138-f6b5ed9611cf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:48.747309 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 107/217] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:48.753125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328b9fa-ea86-497f-b138-f6b5ed9611cf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:48.814364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 111/218] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:48.828955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3cd6c-2114-4cd5-950d-f6b3112b59b1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:48.884627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328b9fa-ea86-497f-b138-f6b5ed9611cf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:48.927776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801e3db-d554-4790-85a7-9a1437001863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:48.927974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801e3db-d554-4790-85a7-9a1437001863 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:48.931028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801e3db-d554-4790-85a7-9a1437001863 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:48.931028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801e3db-d554-4790-85a7-9a1437001863 None admin] PATH_INFO: `/v3/users/95a939dd2ec34598854b36d71abeec51` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:48.947237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801e3db-d554-4790-85a7-9a1437001863 None admin] RBAC: Authorizing `identity:get_user(user_id=95a939dd2ec34598854b36d71abeec51)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:48.950173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801e3db-d554-4790-85a7-9a1437001863 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:48.953250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 108/219] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:48 2022] GET /identity/v3/users/95a939dd2ec34598854b36d71abeec51 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:48.958271 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801e3db-d554-4790-85a7-9a1437001863 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:48.958514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3cd6c-2114-4cd5-950d-f6b3112b59b1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:48.961161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:48.980481 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:49.004812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa4076-b724-4e0d-ab4e-1ace73d94d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:49.006304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa4076-b724-4e0d-ab4e-1ace73d94d2d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:49.006587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa4076-b724-4e0d-ab4e-1ace73d94d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:49.006863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa4076-b724-4e0d-ab4e-1ace73d94d2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:49.032515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa4076-b724-4e0d-ab4e-1ace73d94d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-38aa4076-b724-4e0d-ab4e-1ace73d94d2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:49.040631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:49.041869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:49.042726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:49.043336 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 112/220] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:16:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:16:49.053272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa4076-b724-4e0d-ab4e-1ace73d94d2d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:49.090842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801e3db-d554-4790-85a7-9a1437001863 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:49.135099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6ffcd-09bd-428f-8c86-0e30708f9cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:49.135879 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6ffcd-09bd-428f-8c86-0e30708f9cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:49.136167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6ffcd-09bd-428f-8c86-0e30708f9cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:49.136446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6ffcd-09bd-428f-8c86-0e30708f9cb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:49.191581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa4076-b724-4e0d-ab4e-1ace73d94d2d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:49.222907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6ffcd-09bd-428f-8c86-0e30708f9cb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:49.224516 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6ffcd-09bd-428f-8c86-0e30708f9cb8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:49.237354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe80b-0ce3-4de6-9eb8-f134b584a1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:49.238710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe80b-0ce3-4de6-9eb8-f134b584a1eb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:49.239152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe80b-0ce3-4de6-9eb8-f134b584a1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:49.239753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe80b-0ce3-4de6-9eb8-f134b584a1eb None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:49.241041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe80b-0ce3-4de6-9eb8-f134b584a1eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:49.241520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe80b-0ce3-4de6-9eb8-f134b584a1eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:49.242285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:49.243537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe80b-0ce3-4de6-9eb8-f134b584a1eb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:49.254763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 113/221] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:49.260111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011f480-b2d1-4d89-b111-a2e10e240ba0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:49.260111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011f480-b2d1-4d89-b111-a2e10e240ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:49.260111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011f480-b2d1-4d89-b111-a2e10e240ba0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:49.261518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 114/222] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:49.263671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011f480-b2d1-4d89-b111-a2e10e240ba0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:49.301717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:49.350040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 109/223] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 06:16:49.351784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18b97-bdb2-4cd6-8d46-615a73cdbde7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:49.352279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18b97-bdb2-4cd6-8d46-615a73cdbde7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:49.352757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18b97-bdb2-4cd6-8d46-615a73cdbde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:49.374737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-05a18b97-bdb2-4cd6-8d46-615a73cdbde7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:49.376728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-05a18b97-bdb2-4cd6-8d46-615a73cdbde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:49.394468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011f480-b2d1-4d89-b111-a2e10e240ba0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:49.439725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bba932-9ebd-4403-8f72-b9bfa97c64b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:49.441358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bba932-9ebd-4403-8f72-b9bfa97c64b4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:49.441868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bba932-9ebd-4403-8f72-b9bfa97c64b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:49.442386 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bba932-9ebd-4403-8f72-b9bfa97c64b4 None admin] PATH_INFO: `/v3/users/3875dea0920542f28b6e5f3ba2586b74` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:49.444271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bba932-9ebd-4403-8f72-b9bfa97c64b4 None admin] RBAC: Authorizing `identity:get_user(user_id=3875dea0920542f28b6e5f3ba2586b74)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:49.446717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bba932-9ebd-4403-8f72-b9bfa97c64b4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:49.454446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 115/224] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:49 2022] GET /identity/v3/users/3875dea0920542f28b6e5f3ba2586b74 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:49.454858 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bba932-9ebd-4403-8f72-b9bfa97c64b4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:49.464675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 110/225] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:49.465593 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a18b97-bdb2-4cd6-8d46-615a73cdbde7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:49.524818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:49.584723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:49.594934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bba932-9ebd-4403-8f72-b9bfa97c64b4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:49.600642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a18b97-bdb2-4cd6-8d46-615a73cdbde7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:49.641011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81b33f-01ef-43b6-8206-d8b1a8aa7bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:49.642276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81b33f-01ef-43b6-8206-d8b1a8aa7bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:49.642954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81b33f-01ef-43b6-8206-d8b1a8aa7bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:49.643435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81b33f-01ef-43b6-8206-d8b1a8aa7bd0 None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:49.643983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5febed5-ef6c-4fea-9c98-420376061178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:49.644903 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5febed5-ef6c-4fea-9c98-420376061178 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:49.645119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5febed5-ef6c-4fea-9c98-420376061178 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:49.645395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5febed5-ef6c-4fea-9c98-420376061178 None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:49.645746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81b33f-01ef-43b6-8206-d8b1a8aa7bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:49.646595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5febed5-ef6c-4fea-9c98-420376061178 None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:49.647651 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81b33f-01ef-43b6-8206-d8b1a8aa7bd0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:49.648015 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5febed5-ef6c-4fea-9c98-420376061178 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:49.649814 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 111/226] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:49 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:49.654316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 116/227] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:49 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:49.655426 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5febed5-ef6c-4fea-9c98-420376061178 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:49.658078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81b33f-01ef-43b6-8206-d8b1a8aa7bd0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:49.783297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5febed5-ef6c-4fea-9c98-420376061178 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:49.796838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81b33f-01ef-43b6-8206-d8b1a8aa7bd0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:49.801542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:49.831740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8233c8e-4370-4cdb-917e-668b251dc593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:49.832555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8233c8e-4370-4cdb-917e-668b251dc593 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:49.832837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8233c8e-4370-4cdb-917e-668b251dc593 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:49.833365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8233c8e-4370-4cdb-917e-668b251dc593 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:49.839855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def0eb9-3d9f-447c-8079-a1454c41a728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:49.841000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def0eb9-3d9f-447c-8079-a1454c41a728 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:49.841537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def0eb9-3d9f-447c-8079-a1454c41a728 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:49.842180 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def0eb9-3d9f-447c-8079-a1454c41a728 None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8/users/3875dea0920542f28b6e5f3ba2586b74/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:49.844439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def0eb9-3d9f-447c-8079-a1454c41a728 None admin] RBAC: Authorizing `identity:create_grant(project_id=860106fdc2dc43498b8d25ed76d618e8, user_id=3875dea0920542f28b6e5f3ba2586b74, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:49.844762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8233c8e-4370-4cdb-917e-668b251dc593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:49.846465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def0eb9-3d9f-447c-8079-a1454c41a728 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b8233c8e-4370-4cdb-917e-668b251dc593 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:49.854389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:49.855283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:49.856091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 112/228] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:16:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:49.857305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8233c8e-4370-4cdb-917e-668b251dc593 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:49.865453 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 117/229] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:16:49 2022] PUT /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8/users/3875dea0920542f28b6e5f3ba2586b74/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:16:49.871974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def0eb9-3d9f-447c-8079-a1454c41a728 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:49.912346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:49.972963 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8233c8e-4370-4cdb-917e-668b251dc593 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:50.014139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce25da6-1fd3-4cce-ae11-14c9eadd098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:50.014139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce25da6-1fd3-4cce-ae11-14c9eadd098b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:50.014139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce25da6-1fd3-4cce-ae11-14c9eadd098b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:50.014139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce25da6-1fd3-4cce-ae11-14c9eadd098b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:50.014570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce25da6-1fd3-4cce-ae11-14c9eadd098b None admin] RBAC: Adding query filter params (role.id=d3c75d38368d47d799bf59a0f9c879ea, scope.project.id=860106fdc2dc43498b8d25ed76d618e8, user.id=95a939dd2ec34598854b36d71abeec51) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:50.014570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce25da6-1fd3-4cce-ae11-14c9eadd098b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:50.014570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce25da6-1fd3-4cce-ae11-14c9eadd098b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:50.020941 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 113/230] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:49 2022] GET /identity/v3/role_assignments?user.id=95a939dd2ec34598854b36d71abeec51&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:50.069164 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def0eb9-3d9f-447c-8079-a1454c41a728 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:50.120671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94ae81-6c45-4163-8ec9-5408a35540c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:50.121400 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94ae81-6c45-4163-8ec9-5408a35540c6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:50.121686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94ae81-6c45-4163-8ec9-5408a35540c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:50.121986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94ae81-6c45-4163-8ec9-5408a35540c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:50.122978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94ae81-6c45-4163-8ec9-5408a35540c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:50.123363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94ae81-6c45-4163-8ec9-5408a35540c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:50.124934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94ae81-6c45-4163-8ec9-5408a35540c6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:50.135537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 118/231] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:16:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:50.141688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce25da6-1fd3-4cce-ae11-14c9eadd098b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:50.202951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:50.278986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce25da6-1fd3-4cce-ae11-14c9eadd098b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:50.315276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb0261-ae73-4836-ac1f-4c8691f35f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:50.316160 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb0261-ae73-4836-ac1f-4c8691f35f44 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:50.316443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb0261-ae73-4836-ac1f-4c8691f35f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:50.316717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb0261-ae73-4836-ac1f-4c8691f35f44 None admin] PATH_INFO: `/v3/users/3875dea0920542f28b6e5f3ba2586b74` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:50.317915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb0261-ae73-4836-ac1f-4c8691f35f44 None admin] RBAC: Authorizing `identity:get_user(user_id=3875dea0920542f28b6e5f3ba2586b74)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:50.319368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb0261-ae73-4836-ac1f-4c8691f35f44 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:50.321347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 114/232] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:50 2022] GET /identity/v3/users/3875dea0920542f28b6e5f3ba2586b74 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:50.327398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94ae81-6c45-4163-8ec9-5408a35540c6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:50.471077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:50.474740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94ae81-6c45-4163-8ec9-5408a35540c6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:50.519777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0dc3a-4f32-4fad-925f-d9f428b7432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:50.520664 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0dc3a-4f32-4fad-925f-d9f428b7432c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:50.520953 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0dc3a-4f32-4fad-925f-d9f428b7432c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:50.521235 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0dc3a-4f32-4fad-925f-d9f428b7432c None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:50.522466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0dc3a-4f32-4fad-925f-d9f428b7432c None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:50.523924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0dc3a-4f32-4fad-925f-d9f428b7432c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:50.525896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 119/233] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:50 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:50.531911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb0261-ae73-4836-ac1f-4c8691f35f44 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:50.677099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb0261-ae73-4836-ac1f-4c8691f35f44 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:50.679507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:50.714370 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e0cb0-f2fd-4fb2-af1e-ab397631e36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:50.715274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0cb0-f2fd-4fb2-af1e-ab397631e36b None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:50.715557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0cb0-f2fd-4fb2-af1e-ab397631e36b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:50.715832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0cb0-f2fd-4fb2-af1e-ab397631e36b None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8/users/3875dea0920542f28b6e5f3ba2586b74/roles/d534ab3d0a904f0babeb90d2679d1b86` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:50.718118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e0cb0-f2fd-4fb2-af1e-ab397631e36b None admin] RBAC: Authorizing `identity:create_grant(project_id=860106fdc2dc43498b8d25ed76d618e8, user_id=3875dea0920542f28b6e5f3ba2586b74, role_id=d534ab3d0a904f0babeb90d2679d1b86)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:50.719573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e0cb0-f2fd-4fb2-af1e-ab397631e36b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:50.736152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 115/234] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:16:50 2022] PUT /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8/users/3875dea0920542f28b6e5f3ba2586b74/roles/d534ab3d0a904f0babeb90d2679d1b86 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:16:51.117448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5791e-73f6-4bf4-be91-4bb722834fe5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:51.117788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5791e-73f6-4bf4-be91-4bb722834fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:51.118047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5791e-73f6-4bf4-be91-4bb722834fe5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:51.119479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 120/235] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:51.125308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f602f5-3876-45d0-ac3a-a6d9738ed7a0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:51.125607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f602f5-3876-45d0-ac3a-a6d9738ed7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:51.125890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f602f5-3876-45d0-ac3a-a6d9738ed7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:51.143653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-41f602f5-3876-45d0-ac3a-a6d9738ed7a0 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:51.144836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-41f602f5-3876-45d0-ac3a-a6d9738ed7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:51.230050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 116/236] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:51.268343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f602f5-3876-45d0-ac3a-a6d9738ed7a0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:51.268518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bed58-5744-4431-b43d-967b55e450a2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:51.268809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bed58-5744-4431-b43d-967b55e450a2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:51.269097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bed58-5744-4431-b43d-967b55e450a2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:51.270666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 121/237] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:51.288135 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f016a-310a-45a0-8d4b-a8d657198fe0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:51.288424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f016a-310a-45a0-8d4b-a8d657198fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:51.288947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f016a-310a-45a0-8d4b-a8d657198fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:51.305902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-630f016a-310a-45a0-8d4b-a8d657198fe0 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:51.308358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-630f016a-310a-45a0-8d4b-a8d657198fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:51.420333 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:51.423359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 122/238] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:16:51.443901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f602f5-3876-45d0-ac3a-a6d9738ed7a0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:51.464684 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f016a-310a-45a0-8d4b-a8d657198fe0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:51.503718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6812c1-c906-4749-9e99-6fe9570e5dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:51.504538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6812c1-c906-4749-9e99-6fe9570e5dcb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:51.504824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6812c1-c906-4749-9e99-6fe9570e5dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:51.505102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6812c1-c906-4749-9e99-6fe9570e5dcb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:51.527241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6812c1-c906-4749-9e99-6fe9570e5dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3b6812c1-c906-4749-9e99-6fe9570e5dcb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:51.540161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:51.541784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:51.542609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:51.550070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 117/239] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:51.551202 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6812c1-c906-4749-9e99-6fe9570e5dcb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:51.610371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:51.651381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f016a-310a-45a0-8d4b-a8d657198fe0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:51.709392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be6bfc-8137-4d8c-a9ec-20cdda463818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:51.711178 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be6bfc-8137-4d8c-a9ec-20cdda463818 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:51.711620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be6bfc-8137-4d8c-a9ec-20cdda463818 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:51.712190 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be6bfc-8137-4d8c-a9ec-20cdda463818 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:51.714122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be6bfc-8137-4d8c-a9ec-20cdda463818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:51.715826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be6bfc-8137-4d8c-a9ec-20cdda463818 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:51.717930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 123/240] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:51.729990 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be6bfc-8137-4d8c-a9ec-20cdda463818 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:51.772712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6812c1-c906-4749-9e99-6fe9570e5dcb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:51.820103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ca9de-107f-42e1-b92e-b9195ed7c371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:51.821009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ca9de-107f-42e1-b92e-b9195ed7c371 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:51.821460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ca9de-107f-42e1-b92e-b9195ed7c371 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:51.822453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ca9de-107f-42e1-b92e-b9195ed7c371 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:51.824146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ca9de-107f-42e1-b92e-b9195ed7c371 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:51.824755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ca9de-107f-42e1-b92e-b9195ed7c371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:51.827452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ca9de-107f-42e1-b92e-b9195ed7c371 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:51.843156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 118/241] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:51.851006 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ca9de-107f-42e1-b92e-b9195ed7c371 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:51.852355 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be6bfc-8137-4d8c-a9ec-20cdda463818 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:51.891318 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473177-d100-451c-bd3e-c43bc09e11ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:51.891917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473177-d100-451c-bd3e-c43bc09e11ae None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:51.892195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473177-d100-451c-bd3e-c43bc09e11ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:51.892467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473177-d100-451c-bd3e-c43bc09e11ae None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:51.893179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b473177-d100-451c-bd3e-c43bc09e11ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:51.894663 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b473177-d100-451c-bd3e-c43bc09e11ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:51.895603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:51.921438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 124/242] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:16:51 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:52.002609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ca9de-107f-42e1-b92e-b9195ed7c371 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:52.045192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de285fbc-8e80-4d5b-98a9-b290beafe89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:52.046195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de285fbc-8e80-4d5b-98a9-b290beafe89b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:52.046478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de285fbc-8e80-4d5b-98a9-b290beafe89b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:52.046752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de285fbc-8e80-4d5b-98a9-b290beafe89b None admin] PATH_INFO: `/v3/users/5f636619f809432f9eb1d58743a7f119` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:52.062358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de285fbc-8e80-4d5b-98a9-b290beafe89b None admin] RBAC: Authorizing `identity:get_user(user_id=5f636619f809432f9eb1d58743a7f119)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:52.063834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de285fbc-8e80-4d5b-98a9-b290beafe89b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:52.070044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 119/243] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:51 2022] GET /identity/v3/users/5f636619f809432f9eb1d58743a7f119 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:52.072317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473177-d100-451c-bd3e-c43bc09e11ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:52.082988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:52.192713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473177-d100-451c-bd3e-c43bc09e11ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:52.232936 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f9035-b33b-4936-8717-855516fe400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:52.233603 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9035-b33b-4936-8717-855516fe400c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:52.233895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9035-b33b-4936-8717-855516fe400c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:52.234262 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9035-b33b-4936-8717-855516fe400c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:52.308555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f9035-b33b-4936-8717-855516fe400c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:52.310068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f9035-b33b-4936-8717-855516fe400c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:52.318185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:52.418013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 125/244] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:52.423836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de285fbc-8e80-4d5b-98a9-b290beafe89b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:52.506180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:52.535468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de285fbc-8e80-4d5b-98a9-b290beafe89b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:52.572089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a6821-edf3-4e8c-a308-53a1fa0695df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:52.572967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a6821-edf3-4e8c-a308-53a1fa0695df None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:52.573251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a6821-edf3-4e8c-a308-53a1fa0695df None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:52.573527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a6821-edf3-4e8c-a308-53a1fa0695df None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:52.574781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a6821-edf3-4e8c-a308-53a1fa0695df None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:52.576240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a6821-edf3-4e8c-a308-53a1fa0695df None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:52.582047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 120/245] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:52 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:52.584254 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f9035-b33b-4936-8717-855516fe400c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:52.595698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50652d-af67-478a-8bfc-543f0f31f4c8 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:52.595994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50652d-af67-478a-8bfc-543f0f31f4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:52.596281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50652d-af67-478a-8bfc-543f0f31f4c8 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:52.598201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 121/246] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:52.604000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22d8d4-06f8-4ff1-9d26-2918b379525f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:52.604393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22d8d4-06f8-4ff1-9d26-2918b379525f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:52.604687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22d8d4-06f8-4ff1-9d26-2918b379525f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:52.624348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2c22d8d4-06f8-4ff1-9d26-2918b379525f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:52.625546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2c22d8d4-06f8-4ff1-9d26-2918b379525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:52.719273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 122/247] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:52.734582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f9035-b33b-4936-8717-855516fe400c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:52.743197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:52.743877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22d8d4-06f8-4ff1-9d26-2918b379525f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:52.777342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58538b7c-3fbd-4dbe-920a-b8fa612cc77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:52.778287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58538b7c-3fbd-4dbe-920a-b8fa612cc77a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:52.778715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58538b7c-3fbd-4dbe-920a-b8fa612cc77a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:52.779300 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58538b7c-3fbd-4dbe-920a-b8fa612cc77a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:52.780337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58538b7c-3fbd-4dbe-920a-b8fa612cc77a None admin] RBAC: Adding query filter params (role.id=a889423e7cb541dc99e90f4293116033, scope.project.id=860106fdc2dc43498b8d25ed76d618e8, user.id=5f636619f809432f9eb1d58743a7f119) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:52.781075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58538b7c-3fbd-4dbe-920a-b8fa612cc77a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:52.782977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58538b7c-3fbd-4dbe-920a-b8fa612cc77a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:52.791548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 126/248] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:52 2022] GET /identity/v3/role_assignments?user.id=5f636619f809432f9eb1d58743a7f119&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=a889423e7cb541dc99e90f4293116033 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:52.793164 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088665b-4fcf-41ad-b277-8a74277bde70 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:52.793457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088665b-4fcf-41ad-b277-8a74277bde70 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:52.793740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088665b-4fcf-41ad-b277-8a74277bde70 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:52.796174 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 127/249] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:52.805836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613af51f-2434-49e6-8028-809efeac00f5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:52.806152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613af51f-2434-49e6-8028-809efeac00f5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:52.806428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613af51f-2434-49e6-8028-809efeac00f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:52.835043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-613af51f-2434-49e6-8028-809efeac00f5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:52.836697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-613af51f-2434-49e6-8028-809efeac00f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:52.846630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:52.891879 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22d8d4-06f8-4ff1-9d26-2918b379525f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:52.947551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 128/250] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:52.971789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613af51f-2434-49e6-8028-809efeac00f5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:52.979961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968fdc1-1bde-4364-aab2-a52940647b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:52.980788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968fdc1-1bde-4364-aab2-a52940647b07 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:52.981072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968fdc1-1bde-4364-aab2-a52940647b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:52.981353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968fdc1-1bde-4364-aab2-a52940647b07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:53.005559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968fdc1-1bde-4364-aab2-a52940647b07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1968fdc1-1bde-4364-aab2-a52940647b07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:53.013812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:53.014707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:53.015487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:53.018391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 123/251] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:16:53.025032 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968fdc1-1bde-4364-aab2-a52940647b07 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:53.103119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:53.106586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613af51f-2434-49e6-8028-809efeac00f5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:53.194820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433e5d0-a4d2-4ccf-9484-03612fb05b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:53.195633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433e5d0-a4d2-4ccf-9484-03612fb05b2f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:53.195916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433e5d0-a4d2-4ccf-9484-03612fb05b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:53.196192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433e5d0-a4d2-4ccf-9484-03612fb05b2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:53.203065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968fdc1-1bde-4364-aab2-a52940647b07 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:53.213660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433e5d0-a4d2-4ccf-9484-03612fb05b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4433e5d0-a4d2-4ccf-9484-03612fb05b2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:53.221740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:53.222662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:53.223452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 129/252] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:16:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:16:53.224598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827bf52-f67b-4c81-a259-cfbb4267503f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:53.225968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827bf52-f67b-4c81-a259-cfbb4267503f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:53.225968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827bf52-f67b-4c81-a259-cfbb4267503f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:53.226787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 130/253] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:53.232763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f0827-b8a9-4d07-9c5d-d2d0501cb1df None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:53.233120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f0827-b8a9-4d07-9c5d-d2d0501cb1df None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:53.233416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f0827-b8a9-4d07-9c5d-d2d0501cb1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:53.258981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-176f0827-b8a9-4d07-9c5d-d2d0501cb1df None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:53.260016 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-176f0827-b8a9-4d07-9c5d-d2d0501cb1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:53.266191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:53.277089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5da4e-3a23-4c42-ae2e-f058a72567ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:53.277780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5da4e-3a23-4c42-ae2e-f058a72567ea None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:53.278103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5da4e-3a23-4c42-ae2e-f058a72567ea None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:53.278387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5da4e-3a23-4c42-ae2e-f058a72567ea None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:53.279155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5da4e-3a23-4c42-ae2e-f058a72567ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:53.279467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5da4e-3a23-4c42-ae2e-f058a72567ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:53.280942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5da4e-3a23-4c42-ae2e-f058a72567ea None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:53.300934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 124/254] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:53.301791 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5da4e-3a23-4c42-ae2e-f058a72567ea None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:53.409283 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 131/255] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:53.414133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f0827-b8a9-4d07-9c5d-d2d0501cb1df None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:53.486820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5da4e-3a23-4c42-ae2e-f058a72567ea None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:53.525768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405835b-83bb-4fd3-ae87-f802661b09c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:53.526844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405835b-83bb-4fd3-ae87-f802661b09c9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:53.527342 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405835b-83bb-4fd3-ae87-f802661b09c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:53.527842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405835b-83bb-4fd3-ae87-f802661b09c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:53.528796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405835b-83bb-4fd3-ae87-f802661b09c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:53.529299 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405835b-83bb-4fd3-ae87-f802661b09c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:53.530910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405835b-83bb-4fd3-ae87-f802661b09c9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:53.541601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 125/256] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:53.542795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405835b-83bb-4fd3-ae87-f802661b09c9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:53.582578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:53.607069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f0827-b8a9-4d07-9c5d-d2d0501cb1df None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:53.614836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:53.656134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405835b-83bb-4fd3-ae87-f802661b09c9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:53.691415 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c552423-c33d-4e47-bad4-03088160e263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:53.692566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c552423-c33d-4e47-bad4-03088160e263 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:53.693056 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c552423-c33d-4e47-bad4-03088160e263 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:53.693528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c552423-c33d-4e47-bad4-03088160e263 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:53.705660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c552423-c33d-4e47-bad4-03088160e263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:53.710819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5957180-7fcb-4164-bcd1-a76b8672b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:53.712340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5957180-7fcb-4164-bcd1-a76b8672b672 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:53.712745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5957180-7fcb-4164-bcd1-a76b8672b672 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:53.713166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5957180-7fcb-4164-bcd1-a76b8672b672 None admin] PATH_INFO: `/v3/users/95a939dd2ec34598854b36d71abeec51` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:53.718711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5957180-7fcb-4164-bcd1-a76b8672b672 None admin] RBAC: Authorizing `identity:get_user(user_id=95a939dd2ec34598854b36d71abeec51)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:53.720195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5957180-7fcb-4164-bcd1-a76b8672b672 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3c552423-c33d-4e47-bad4-03088160e263 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:53.720667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:53.721528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:16:53.722670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:53.723269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 132/257] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:53 2022] GET /identity/v3/users/95a939dd2ec34598854b36d71abeec51 => generated 297 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:53.723269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5957180-7fcb-4164-bcd1-a76b8672b672 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:53.724987 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 126/258] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:16:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:53.731173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c552423-c33d-4e47-bad4-03088160e263 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:53.839228 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c552423-c33d-4e47-bad4-03088160e263 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:53.874294 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:53.879840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ebd3c-6b00-4132-9afe-d45a52c901c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:53.880940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ebd3c-6b00-4132-9afe-d45a52c901c9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:53.881388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ebd3c-6b00-4132-9afe-d45a52c901c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:53.884028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ebd3c-6b00-4132-9afe-d45a52c901c9 None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:53.888925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ebd3c-6b00-4132-9afe-d45a52c901c9 None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:53.893338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ebd3c-6b00-4132-9afe-d45a52c901c9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:53.897542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 127/259] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:53 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:53.898543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ebd3c-6b00-4132-9afe-d45a52c901c9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:53.944536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5957180-7fcb-4164-bcd1-a76b8672b672 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:54.016524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d66bb-1453-4067-817c-9c9780cd33af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:54.019216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d66bb-1453-4067-817c-9c9780cd33af None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:54.020592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d66bb-1453-4067-817c-9c9780cd33af None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:54.021053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d66bb-1453-4067-817c-9c9780cd33af None admin] PATH_INFO: `/v3/users/3875dea0920542f28b6e5f3ba2586b74` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:54.024506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d66bb-1453-4067-817c-9c9780cd33af None admin] RBAC: Authorizing `identity:get_user(user_id=3875dea0920542f28b6e5f3ba2586b74)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:54.026189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d66bb-1453-4067-817c-9c9780cd33af None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:54.029805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 133/260] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:53 2022] GET /identity/v3/users/3875dea0920542f28b6e5f3ba2586b74 => generated 283 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:54.033113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d66bb-1453-4067-817c-9c9780cd33af None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:54.082390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:54.147963 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ebd3c-6b00-4132-9afe-d45a52c901c9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:54.157085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d66bb-1453-4067-817c-9c9780cd33af None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:54.198915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2266b6-29bd-4974-937d-bfabae51277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:54.200206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2266b6-29bd-4974-937d-bfabae51277e None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:54.200686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2266b6-29bd-4974-937d-bfabae51277e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:54.201306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2266b6-29bd-4974-937d-bfabae51277e None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8/users/95a939dd2ec34598854b36d71abeec51/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:54.203610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2266b6-29bd-4974-937d-bfabae51277e None admin] RBAC: Authorizing `identity:create_grant(project_id=860106fdc2dc43498b8d25ed76d618e8, user_id=95a939dd2ec34598854b36d71abeec51, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:54.205294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2266b6-29bd-4974-937d-bfabae51277e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:54.227831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 134/261] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:16:54 2022] PUT /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8/users/95a939dd2ec34598854b36d71abeec51/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:16:54.229050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2266b6-29bd-4974-937d-bfabae51277e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:54.230044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cab4e-bf4f-4f9a-b839-541923f30b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:54.231436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cab4e-bf4f-4f9a-b839-541923f30b85 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:54.233053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cab4e-bf4f-4f9a-b839-541923f30b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:54.233529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cab4e-bf4f-4f9a-b839-541923f30b85 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:54.235725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7cab4e-bf4f-4f9a-b839-541923f30b85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:54.237158 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7cab4e-bf4f-4f9a-b839-541923f30b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:54.238827 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7cab4e-bf4f-4f9a-b839-541923f30b85 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:54.259986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 128/262] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:16:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:54.265693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cab4e-bf4f-4f9a-b839-541923f30b85 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:54.366190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:54.419514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:54.434206 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cab4e-bf4f-4f9a-b839-541923f30b85 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:54.456271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2266b6-29bd-4974-937d-bfabae51277e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:54.476027 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a2b24-2007-4adb-9779-ab1325db62cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:54.477840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a2b24-2007-4adb-9779-ab1325db62cf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:54.478388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a2b24-2007-4adb-9779-ab1325db62cf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:54.478890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a2b24-2007-4adb-9779-ab1325db62cf None admin] PATH_INFO: `/v3/users/3875dea0920542f28b6e5f3ba2586b74` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:54.480420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a2b24-2007-4adb-9779-ab1325db62cf None admin] RBAC: Authorizing `identity:get_user(user_id=3875dea0920542f28b6e5f3ba2586b74)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:54.482481 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a2b24-2007-4adb-9779-ab1325db62cf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:54.484952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 129/263] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:54 2022] GET /identity/v3/users/3875dea0920542f28b6e5f3ba2586b74 => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:54.490337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a2b24-2007-4adb-9779-ab1325db62cf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:54.543597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03506b3e-30bf-4d68-97cf-2f595469c47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:54.544282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03506b3e-30bf-4d68-97cf-2f595469c47a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:54.544573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03506b3e-30bf-4d68-97cf-2f595469c47a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:54.544854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03506b3e-30bf-4d68-97cf-2f595469c47a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:54.607213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a2b24-2007-4adb-9779-ab1325db62cf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:54.623197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:54.646338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435bb73-c6b9-4238-9227-8e10db0a4949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:54.647342 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435bb73-c6b9-4238-9227-8e10db0a4949 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:54.647839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435bb73-c6b9-4238-9227-8e10db0a4949 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:54.648313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435bb73-c6b9-4238-9227-8e10db0a4949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:54.724275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435bb73-c6b9-4238-9227-8e10db0a4949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:54.726272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435bb73-c6b9-4238-9227-8e10db0a4949 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:54.788795 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03506b3e-30bf-4d68-97cf-2f595469c47a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:54.791532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03506b3e-30bf-4d68-97cf-2f595469c47a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:54.832947 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 130/264] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 06:16:54.838652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435bb73-c6b9-4238-9227-8e10db0a4949 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:54.862346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:54.962878 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435bb73-c6b9-4238-9227-8e10db0a4949 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:55.009004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d663a-a04f-43e2-8cea-3932f3ede4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:55.010650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d663a-a04f-43e2-8cea-3932f3ede4fa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:55.011160 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d663a-a04f-43e2-8cea-3932f3ede4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:55.011657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d663a-a04f-43e2-8cea-3932f3ede4fa None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:55.013098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d663a-a04f-43e2-8cea-3932f3ede4fa None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:55.014928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d663a-a04f-43e2-8cea-3932f3ede4fa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:55.016858 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 131/265] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:54 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:55.022590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d663a-a04f-43e2-8cea-3932f3ede4fa None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:55.126331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:55.159976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:55.174166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 135/266] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 945 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:55.182903 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03506b3e-30bf-4d68-97cf-2f595469c47a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:55.194998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d663a-a04f-43e2-8cea-3932f3ede4fa None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:55.270481 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16fcf5-b7a3-4714-b16c-c006dbec5642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:55.271062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16fcf5-b7a3-4714-b16c-c006dbec5642 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:55.271350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16fcf5-b7a3-4714-b16c-c006dbec5642 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:55.271687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16fcf5-b7a3-4714-b16c-c006dbec5642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:55.273458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16fcf5-b7a3-4714-b16c-c006dbec5642 None admin] RBAC: Adding query filter params (role.id=d534ab3d0a904f0babeb90d2679d1b86, scope.project.id=860106fdc2dc43498b8d25ed76d618e8, user.id=3875dea0920542f28b6e5f3ba2586b74) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:55.273874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16fcf5-b7a3-4714-b16c-c006dbec5642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:55.276441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16fcf5-b7a3-4714-b16c-c006dbec5642 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:55.289811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 132/267] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:55 2022] GET /identity/v3/role_assignments?user.id=3875dea0920542f28b6e5f3ba2586b74&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=d534ab3d0a904f0babeb90d2679d1b86 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:55.343860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03506b3e-30bf-4d68-97cf-2f595469c47a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:55.402324 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abfd58a-8e00-4a35-8dbb-6254f0e06c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:55.403593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfd58a-8e00-4a35-8dbb-6254f0e06c6c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:55.404076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfd58a-8e00-4a35-8dbb-6254f0e06c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:55.404548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfd58a-8e00-4a35-8dbb-6254f0e06c6c None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:55.410908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfd58a-8e00-4a35-8dbb-6254f0e06c6c None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:55.412781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfd58a-8e00-4a35-8dbb-6254f0e06c6c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:55.415676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 136/268] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:55 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:55.421732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16fcf5-b7a3-4714-b16c-c006dbec5642 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:55.546066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16fcf5-b7a3-4714-b16c-c006dbec5642 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:55.560035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf5769-c66c-4469-978d-7e462045b9ff None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:55.560657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf5769-c66c-4469-978d-7e462045b9ff None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:55.561215 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf5769-c66c-4469-978d-7e462045b9ff None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:55.567103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 137/269] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:16:55.568675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1a12d-db70-429b-823f-328d12cd681e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:55.568966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1a12d-db70-429b-823f-328d12cd681e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:55.569250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1a12d-db70-429b-823f-328d12cd681e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:55.577995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 138/270] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:55.579776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f9340-6122-4d80-a03b-521c7cf8f793 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:55.580072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f9340-6122-4d80-a03b-521c7cf8f793 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:55.580353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f9340-6122-4d80-a03b-521c7cf8f793 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:55.586207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 139/271] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:55.587655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee4231-6041-45d0-80bd-91d907a2f829 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:55.587941 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee4231-6041-45d0-80bd-91d907a2f829 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:55.588218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee4231-6041-45d0-80bd-91d907a2f829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:55.591474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acf8f8-dab6-4f41-a691-5be25b9fd345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:55.592142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acf8f8-dab6-4f41-a691-5be25b9fd345 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:55.592424 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acf8f8-dab6-4f41-a691-5be25b9fd345 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:55.592696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acf8f8-dab6-4f41-a691-5be25b9fd345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:55.593545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acf8f8-dab6-4f41-a691-5be25b9fd345 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=860106fdc2dc43498b8d25ed76d618e8, user.id=3875dea0920542f28b6e5f3ba2586b74) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:55.593856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acf8f8-dab6-4f41-a691-5be25b9fd345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:55.595980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acf8f8-dab6-4f41-a691-5be25b9fd345 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:55.608220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 133/272] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:55 2022] GET /identity/v3/role_assignments?user.id=3875dea0920542f28b6e5f3ba2586b74&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:55.608220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bdbb6-b36f-49e7-b694-fa80e852a2dc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:55.608220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bdbb6-b36f-49e7-b694-fa80e852a2dc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:55.608220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bdbb6-b36f-49e7-b694-fa80e852a2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:55.617081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6eee4231-6041-45d0-80bd-91d907a2f829 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:55.620385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6eee4231-6041-45d0-80bd-91d907a2f829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:55.634128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-236bdbb6-b36f-49e7-b694-fa80e852a2dc None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:55.635658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-236bdbb6-b36f-49e7-b694-fa80e852a2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:55.640325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:55.691959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:55.716595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 140/273] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:55.719232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bef1fe-1e9b-47fa-9473-6268a0aec68f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:55.719520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bef1fe-1e9b-47fa-9473-6268a0aec68f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:55.719798 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bef1fe-1e9b-47fa-9473-6268a0aec68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:55.734971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 134/274] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:55.748114 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-65bef1fe-1e9b-47fa-9473-6268a0aec68f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:55.748594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-65bef1fe-1e9b-47fa-9473-6268a0aec68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:55.749692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bdbb6-b36f-49e7-b694-fa80e852a2dc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:55.855278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 141/275] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:55.855278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bef1fe-1e9b-47fa-9473-6268a0aec68f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:55.903832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bdbb6-b36f-49e7-b694-fa80e852a2dc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:55.955926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fb516-6b4c-4ae5-b451-c6d50b022a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:55.956738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fb516-6b4c-4ae5-b451-c6d50b022a37 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:55.957017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fb516-6b4c-4ae5-b451-c6d50b022a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:55.957286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fb516-6b4c-4ae5-b451-c6d50b022a37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:55.969465 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:55.974035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fb516-6b4c-4ae5-b451-c6d50b022a37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4c2fb516-6b4c-4ae5-b451-c6d50b022a37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:55.989282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:55.990300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:55.991090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:55.991679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 135/276] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:55.992711 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fb516-6b4c-4ae5-b451-c6d50b022a37 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:56.017968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bef1fe-1e9b-47fa-9473-6268a0aec68f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:56.036848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:56.069759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ce1d3-8655-419a-bd03-821c0a11a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:56.071956 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ce1d3-8655-419a-bd03-821c0a11a369 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:56.072243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ce1d3-8655-419a-bd03-821c0a11a369 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:56.072910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ce1d3-8655-419a-bd03-821c0a11a369 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:56.074017 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ce1d3-8655-419a-bd03-821c0a11a369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:56.075685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ce1d3-8655-419a-bd03-821c0a11a369 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:56.077598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 142/277] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:56.082863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ce1d3-8655-419a-bd03-821c0a11a369 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:56.119479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fb516-6b4c-4ae5-b451-c6d50b022a37 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:56.160306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f348f9-c651-4bff-878c-489749e6b2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:56.161119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f348f9-c651-4bff-878c-489749e6b2cc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:56.161404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f348f9-c651-4bff-878c-489749e6b2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:56.161680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f348f9-c651-4bff-878c-489749e6b2cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:56.178547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f348f9-c651-4bff-878c-489749e6b2cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a5f348f9-c651-4bff-878c-489749e6b2cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:56.187035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:56.188334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 136/278] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:56.189547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f348f9-c651-4bff-878c-489749e6b2cc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:56.209779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ce1d3-8655-419a-bd03-821c0a11a369 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:56.253265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a23e1-8e74-47b7-a4b9-21c996aadbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:56.253962 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a23e1-8e74-47b7-a4b9-21c996aadbaf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:56.254247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a23e1-8e74-47b7-a4b9-21c996aadbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:56.254511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a23e1-8e74-47b7-a4b9-21c996aadbaf None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:56.255265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a23e1-8e74-47b7-a4b9-21c996aadbaf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:56.255571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a23e1-8e74-47b7-a4b9-21c996aadbaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:56.256949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a23e1-8e74-47b7-a4b9-21c996aadbaf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:56.263654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:56.268923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 143/279] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:56.269499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a23e1-8e74-47b7-a4b9-21c996aadbaf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:56.315637 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f348f9-c651-4bff-878c-489749e6b2cc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:56.325332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:56.356895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f7dfb-77f8-4f1e-a940-c926b44fa4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:56.357863 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f7dfb-77f8-4f1e-a940-c926b44fa4f9 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:56.358394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f7dfb-77f8-4f1e-a940-c926b44fa4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:56.358899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f7dfb-77f8-4f1e-a940-c926b44fa4f9 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:56.359735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f7dfb-77f8-4f1e-a940-c926b44fa4f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:56.361206 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f7dfb-77f8-4f1e-a940-c926b44fa4f9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:56.392959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a23e1-8e74-47b7-a4b9-21c996aadbaf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:56.393134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 137/280] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:16:56 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:56.394803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f7dfb-77f8-4f1e-a940-c926b44fa4f9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:56.454718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37fce4-4ec9-4177-bcc7-257f9b30351e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:56.456553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37fce4-4ec9-4177-bcc7-257f9b30351e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:56.457050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37fce4-4ec9-4177-bcc7-257f9b30351e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:56.458051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37fce4-4ec9-4177-bcc7-257f9b30351e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:56.459612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37fce4-4ec9-4177-bcc7-257f9b30351e None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:56.460454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37fce4-4ec9-4177-bcc7-257f9b30351e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:56.462997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37fce4-4ec9-4177-bcc7-257f9b30351e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:56.477508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 144/281] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:56.485263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37fce4-4ec9-4177-bcc7-257f9b30351e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:56.541841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f7dfb-77f8-4f1e-a940-c926b44fa4f9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:56.565789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:56.586894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f691e4d-e398-42b8-a422-700c4d0d2ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:56.587783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f691e4d-e398-42b8-a422-700c4d0d2ede None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:56.588069 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f691e4d-e398-42b8-a422-700c4d0d2ede None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:56.588349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f691e4d-e398-42b8-a422-700c4d0d2ede None admin] PATH_INFO: `/v3/users/5f636619f809432f9eb1d58743a7f119` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:56.589575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f691e4d-e398-42b8-a422-700c4d0d2ede None admin] RBAC: Authorizing `identity:get_user(user_id=5f636619f809432f9eb1d58743a7f119)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:56.591527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f691e4d-e398-42b8-a422-700c4d0d2ede None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:56.593204 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 138/282] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:56 2022] GET /identity/v3/users/5f636619f809432f9eb1d58743a7f119 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:56.598217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f691e4d-e398-42b8-a422-700c4d0d2ede None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:56.630938 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37fce4-4ec9-4177-bcc7-257f9b30351e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:56.640501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:56.688449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc844f-01d6-4e78-a7a2-11aedc2a9933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:56.689276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc844f-01d6-4e78-a7a2-11aedc2a9933 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:56.689602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc844f-01d6-4e78-a7a2-11aedc2a9933 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:56.689881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc844f-01d6-4e78-a7a2-11aedc2a9933 None admin] PATH_INFO: `/v3/users/d2c9581524014ff6b6fccab34ed5448f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:56.705904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc844f-01d6-4e78-a7a2-11aedc2a9933 None admin] RBAC: Authorizing `identity:get_user(user_id=d2c9581524014ff6b6fccab34ed5448f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:56.707656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc844f-01d6-4e78-a7a2-11aedc2a9933 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:56.709505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 145/283] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:56 2022] GET /identity/v3/users/d2c9581524014ff6b6fccab34ed5448f => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:56.722398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc844f-01d6-4e78-a7a2-11aedc2a9933 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:56.750421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f691e4d-e398-42b8-a422-700c4d0d2ede None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:56.796585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef323e12-19ca-4d81-8221-5726df8a95ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:56.797608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef323e12-19ca-4d81-8221-5726df8a95ff None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:56.798051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef323e12-19ca-4d81-8221-5726df8a95ff None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:56.798402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef323e12-19ca-4d81-8221-5726df8a95ff None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:56.799845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef323e12-19ca-4d81-8221-5726df8a95ff None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:56.801116 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef323e12-19ca-4d81-8221-5726df8a95ff None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:56.803399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 139/284] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:56 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:56.809839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef323e12-19ca-4d81-8221-5726df8a95ff None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:56.858319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc844f-01d6-4e78-a7a2-11aedc2a9933 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:56.882338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:56.908358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f02271-cabb-4ed0-b2b3-26966d5ba3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:56.909020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02271-cabb-4ed0-b2b3-26966d5ba3ed None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:56.909302 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02271-cabb-4ed0-b2b3-26966d5ba3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:56.909578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02271-cabb-4ed0-b2b3-26966d5ba3ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:56.910458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f02271-cabb-4ed0-b2b3-26966d5ba3ed None admin] RBAC: Adding query filter params (role.id=d3c75d38368d47d799bf59a0f9c879ea, user.id=d2c9581524014ff6b6fccab34ed5448f, scope.system=all) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:56.911212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f02271-cabb-4ed0-b2b3-26966d5ba3ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:56.912812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f02271-cabb-4ed0-b2b3-26966d5ba3ed None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:56.919340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 146/285] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jul 21 06:16:56 2022] GET /identity/v3/role_assignments?user.id=d2c9581524014ff6b6fccab34ed5448f&scope.system=all&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:16:56.973515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef323e12-19ca-4d81-8221-5726df8a95ff None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:57.015870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e32f1-4161-4272-bcd2-e72e0d468da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:57.016812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e32f1-4161-4272-bcd2-e72e0d468da6 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:57.017103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e32f1-4161-4272-bcd2-e72e0d468da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:57.017384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e32f1-4161-4272-bcd2-e72e0d468da6 None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8/users/5f636619f809432f9eb1d58743a7f119/roles/a889423e7cb541dc99e90f4293116033` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:57.024112 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e32f1-4161-4272-bcd2-e72e0d468da6 None admin] RBAC: Authorizing `identity:create_grant(project_id=860106fdc2dc43498b8d25ed76d618e8, user_id=5f636619f809432f9eb1d58743a7f119, role_id=a889423e7cb541dc99e90f4293116033)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:57.025814 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e32f1-4161-4272-bcd2-e72e0d468da6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:57.060817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 140/286] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:16:56 2022] PUT /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8/users/5f636619f809432f9eb1d58743a7f119/roles/a889423e7cb541dc99e90f4293116033 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:16:57.124332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509de3f-b5c3-415d-80c3-df108891d061 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:57.124631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509de3f-b5c3-415d-80c3-df108891d061 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:57.124909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509de3f-b5c3-415d-80c3-df108891d061 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:57.126821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 147/287] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:57.135679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34889f-6ee1-486c-ae74-c185c33dea0f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:57.136272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34889f-6ee1-486c-ae74-c185c33dea0f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:57.136734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34889f-6ee1-486c-ae74-c185c33dea0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:57.161552 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0f34889f-6ee1-486c-ae74-c185c33dea0f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:57.162952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0f34889f-6ee1-486c-ae74-c185c33dea0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:57.265596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 141/288] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:16:57.298454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509de3f-b5c3-415d-80c3-df108891d061 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:57.334932 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:57.421885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509de3f-b5c3-415d-80c3-df108891d061 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:57.461625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e015238-6eca-401d-9b5a-585c3e40db40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:57.462472 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e015238-6eca-401d-9b5a-585c3e40db40 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:57.462757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e015238-6eca-401d-9b5a-585c3e40db40 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:57.463036 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e015238-6eca-401d-9b5a-585c3e40db40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:57.479305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e015238-6eca-401d-9b5a-585c3e40db40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4e015238-6eca-401d-9b5a-585c3e40db40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:57.487315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:57.488158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:16:57.488937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 148/289] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:57.495803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34889f-6ee1-486c-ae74-c185c33dea0f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:57.570353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:57.642019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34889f-6ee1-486c-ae74-c185c33dea0f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:57.688921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfba55-4f8b-445d-942a-1a1d6078d419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:57.689600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfba55-4f8b-445d-942a-1a1d6078d419 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:57.689888 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfba55-4f8b-445d-942a-1a1d6078d419 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:57.690209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfba55-4f8b-445d-942a-1a1d6078d419 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:57.691042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfba55-4f8b-445d-942a-1a1d6078d419 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:57.691356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfba55-4f8b-445d-942a-1a1d6078d419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:57.692793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfba55-4f8b-445d-942a-1a1d6078d419 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:57.705165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 142/290] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:57.711744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e015238-6eca-401d-9b5a-585c3e40db40 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:57.827912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:57.841244 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e015238-6eca-401d-9b5a-585c3e40db40 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:57.893412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8c16a-eaf5-4b2d-9520-1b2951abb15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:57.894323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8c16a-eaf5-4b2d-9520-1b2951abb15e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:57.894608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8c16a-eaf5-4b2d-9520-1b2951abb15e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:57.894884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8c16a-eaf5-4b2d-9520-1b2951abb15e None admin] PATH_INFO: `/v3/users/95a939dd2ec34598854b36d71abeec51` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:57.896100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8c16a-eaf5-4b2d-9520-1b2951abb15e None admin] RBAC: Authorizing `identity:get_user(user_id=95a939dd2ec34598854b36d71abeec51)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:57.897553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8c16a-eaf5-4b2d-9520-1b2951abb15e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:57.900704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 149/291] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:57 2022] GET /identity/v3/users/95a939dd2ec34598854b36d71abeec51 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:57.905714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfba55-4f8b-445d-942a-1a1d6078d419 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:58.061761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfba55-4f8b-445d-942a-1a1d6078d419 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:58.066977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:58.104448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ce6f3-2765-4a65-9e65-e5177dcc95e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:58.105033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ce6f3-2765-4a65-9e65-e5177dcc95e8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:58.105319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ce6f3-2765-4a65-9e65-e5177dcc95e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:58.105603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ce6f3-2765-4a65-9e65-e5177dcc95e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:58.175169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ce6f3-2765-4a65-9e65-e5177dcc95e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:58.176651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ce6f3-2765-4a65-9e65-e5177dcc95e8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:58.300531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 143/292] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:16:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:16:58.306914 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8c16a-eaf5-4b2d-9520-1b2951abb15e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:58.447727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8c16a-eaf5-4b2d-9520-1b2951abb15e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:58.494592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a5947-05fc-43a4-8493-d4cb5a98b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:58.495469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a5947-05fc-43a4-8493-d4cb5a98b6ce None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:58.495760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a5947-05fc-43a4-8493-d4cb5a98b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:58.496045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a5947-05fc-43a4-8493-d4cb5a98b6ce None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:58.497255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a5947-05fc-43a4-8493-d4cb5a98b6ce None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:58.498726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a5947-05fc-43a4-8493-d4cb5a98b6ce None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:58.500728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 150/293] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:16:58 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:58.506396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ce6f3-2765-4a65-9e65-e5177dcc95e8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:58.511025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:58.624463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ce6f3-2765-4a65-9e65-e5177dcc95e8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:58.658630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937390a6-bef2-4166-9dbc-b7ef98b6f9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:58.659292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937390a6-bef2-4166-9dbc-b7ef98b6f9cc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:58.659581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937390a6-bef2-4166-9dbc-b7ef98b6f9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:58.659859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937390a6-bef2-4166-9dbc-b7ef98b6f9cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:58.660706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937390a6-bef2-4166-9dbc-b7ef98b6f9cc None admin] RBAC: Adding query filter params (role.id=d3c75d38368d47d799bf59a0f9c879ea, scope.project.id=860106fdc2dc43498b8d25ed76d618e8, user.id=95a939dd2ec34598854b36d71abeec51) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:58.661021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937390a6-bef2-4166-9dbc-b7ef98b6f9cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:58.662512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937390a6-bef2-4166-9dbc-b7ef98b6f9cc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:58.673388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 144/294] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:16:58 2022] GET /identity/v3/role_assignments?user.id=95a939dd2ec34598854b36d71abeec51&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:59.221600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47857aa-500f-4375-a2d3-d04fdee27a8c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.222156 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47857aa-500f-4375-a2d3-d04fdee27a8c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.222236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47857aa-500f-4375-a2d3-d04fdee27a8c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.223837 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 151/295] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:59.230011 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510fb5f-12cb-469f-904a-892ce78cabf7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.230579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510fb5f-12cb-469f-904a-892ce78cabf7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.231080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510fb5f-12cb-469f-904a-892ce78cabf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.251147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5510fb5f-12cb-469f-904a-892ce78cabf7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:59.252666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5510fb5f-12cb-469f-904a-892ce78cabf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:59.264706 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9797d-1a4c-4a00-9749-81a269326f44 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.265310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9797d-1a4c-4a00-9749-81a269326f44 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.265812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9797d-1a4c-4a00-9749-81a269326f44 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.267685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 152/296] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:16:59.274315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747abe58-d28d-4ee0-b88f-ab7fac23465d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.274847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747abe58-d28d-4ee0-b88f-ab7fac23465d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.275297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747abe58-d28d-4ee0-b88f-ab7fac23465d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.297717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-747abe58-d28d-4ee0-b88f-ab7fac23465d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:59.300363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-747abe58-d28d-4ee0-b88f-ab7fac23465d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:59.333185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:59.333984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:59.342911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 145/297] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:16:59.368770 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510fb5f-12cb-469f-904a-892ce78cabf7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:59.405526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 153/298] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:16:59.407294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba250d6-969c-4220-bcb8-c08dbf74cc2c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.407610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba250d6-969c-4220-bcb8-c08dbf74cc2c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.407874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba250d6-969c-4220-bcb8-c08dbf74cc2c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.409270 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 154/299] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:59.414711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a034b5-bf05-4b96-9cda-150a7ff431ed None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.415217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a034b5-bf05-4b96-9cda-150a7ff431ed None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.415651 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a034b5-bf05-4b96-9cda-150a7ff431ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.433610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-14a034b5-bf05-4b96-9cda-150a7ff431ed None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:16:59.435067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-14a034b5-bf05-4b96-9cda-150a7ff431ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:16:59.479945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510fb5f-12cb-469f-904a-892ce78cabf7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:59.512220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 155/300] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:16:59.515079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a034b5-bf05-4b96-9cda-150a7ff431ed None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:59.516915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447348af-cc6d-47eb-9c5d-519b8f41f2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:59.518044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447348af-cc6d-47eb-9c5d-519b8f41f2de None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.518476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447348af-cc6d-47eb-9c5d-519b8f41f2de None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.519019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447348af-cc6d-47eb-9c5d-519b8f41f2de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.530576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447348af-cc6d-47eb-9c5d-519b8f41f2de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-447348af-cc6d-47eb-9c5d-519b8f41f2de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:59.539514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:59.540453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 146/301] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:16:59.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447348af-cc6d-47eb-9c5d-519b8f41f2de None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:59.601969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:59.618639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:59.636382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a034b5-bf05-4b96-9cda-150a7ff431ed None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:59.646474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447348af-cc6d-47eb-9c5d-519b8f41f2de None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:59.677874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c95a97-36e3-457c-869a-eeda1b56044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:59.680079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95a97-36e3-457c-869a-eeda1b56044e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.680504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95a97-36e3-457c-869a-eeda1b56044e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.681023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95a97-36e3-457c-869a-eeda1b56044e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.682437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c95a97-36e3-457c-869a-eeda1b56044e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:59.683895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c95a97-36e3-457c-869a-eeda1b56044e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:59.684507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ce3d8-4903-4334-864b-e3bebb91a14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:59.685318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ce3d8-4903-4334-864b-e3bebb91a14b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.689981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ce3d8-4903-4334-864b-e3bebb91a14b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.690109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 156/302] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:16:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:59.690198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ce3d8-4903-4334-864b-e3bebb91a14b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.690299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c95a97-36e3-457c-869a-eeda1b56044e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:59.704958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ce3d8-4903-4334-864b-e3bebb91a14b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-de4ce3d8-4903-4334-864b-e3bebb91a14b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:16:59.716135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:16:59.717041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:16:59.717804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:16:59.718923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 147/303] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:16:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:16:59.719491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ce3d8-4903-4334-864b-e3bebb91a14b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:59.799093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c95a97-36e3-457c-869a-eeda1b56044e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:59.825738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ce3d8-4903-4334-864b-e3bebb91a14b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:16:59.833859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa729e8-2841-47e7-a4b3-3a4452a2962b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:59.834561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa729e8-2841-47e7-a4b3-3a4452a2962b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.835651 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa729e8-2841-47e7-a4b3-3a4452a2962b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.835781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa729e8-2841-47e7-a4b3-3a4452a2962b None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.836431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa729e8-2841-47e7-a4b3-3a4452a2962b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:16:59.836742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa729e8-2841-47e7-a4b3-3a4452a2962b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:59.838144 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa729e8-2841-47e7-a4b3-3a4452a2962b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:59.842968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:59.846354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 157/304] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:16:59.847297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa729e8-2841-47e7-a4b3-3a4452a2962b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:59.862656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:16:59.865240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0725f6-d260-4d77-b64b-6ec3555f0fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:16:59.865979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0725f6-d260-4d77-b64b-6ec3555f0fa6 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.866275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0725f6-d260-4d77-b64b-6ec3555f0fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.866556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0725f6-d260-4d77-b64b-6ec3555f0fa6 None admin] PATH_INFO: `/v3/groups` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.867254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0725f6-d260-4d77-b64b-6ec3555f0fa6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:16:59.868694 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0725f6-d260-4d77-b64b-6ec3555f0fa6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:16:59.884262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 148/305] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Jul 21 06:16:59 2022] POST /identity/v3/groups => generated 229 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:16:59.885599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c983b-2668-4073-8b66-31c802a09906 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:16:59.885870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c983b-2668-4073-8b66-31c802a09906 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:16:59.886176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c983b-2668-4073-8b66-31c802a09906 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:16:59.887441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 149/306] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:16:59.888165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c983b-2668-4073-8b66-31c802a09906 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:16:59.968928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa729e8-2841-47e7-a4b3-3a4452a2962b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:00.006947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b32872-a801-46ad-8536-94fbca2a4dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:00.007633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b32872-a801-46ad-8536-94fbca2a4dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:00.007920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b32872-a801-46ad-8536-94fbca2a4dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:00.008200 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b32872-a801-46ad-8536-94fbca2a4dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:00.008961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b32872-a801-46ad-8536-94fbca2a4dc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:00.009272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b32872-a801-46ad-8536-94fbca2a4dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:00.010733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b32872-a801-46ad-8536-94fbca2a4dc2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:00.014276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c983b-2668-4073-8b66-31c802a09906 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:00.020231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 158/307] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:16:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:00.020231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1158e-5911-48c1-8b0b-3b16b88f8a1c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:00.020231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1158e-5911-48c1-8b0b-3b16b88f8a1c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:00.022802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1158e-5911-48c1-8b0b-3b16b88f8a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:00.042989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-05f1158e-5911-48c1-8b0b-3b16b88f8a1c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:00.043173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-05f1158e-5911-48c1-8b0b-3b16b88f8a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:00.066864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857148ec-78d3-4849-90ab-3dcfb9d21b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:00.067953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857148ec-78d3-4849-90ab-3dcfb9d21b31 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:00.068384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857148ec-78d3-4849-90ab-3dcfb9d21b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:00.068818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857148ec-78d3-4849-90ab-3dcfb9d21b31 None admin] PATH_INFO: `/v3/users/4c7ac0f524d840b683adacc2faf05adb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:00.086661 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857148ec-78d3-4849-90ab-3dcfb9d21b31 None admin] RBAC: Authorizing `identity:get_user(user_id=4c7ac0f524d840b683adacc2faf05adb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:00.088149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857148ec-78d3-4849-90ab-3dcfb9d21b31 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:00.090127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 150/308] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:16:59 2022] GET /identity/v3/users/4c7ac0f524d840b683adacc2faf05adb => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:00.090739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857148ec-78d3-4849-90ab-3dcfb9d21b31 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:00.130172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 159/309] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:00.132062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:00.132310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1158e-5911-48c1-8b0b-3b16b88f8a1c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:00.142089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:00.204576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857148ec-78d3-4849-90ab-3dcfb9d21b31 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:00.238878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1158e-5911-48c1-8b0b-3b16b88f8a1c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:00.240772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6883a-28bd-4613-a314-3166234a2fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:00.241797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6883a-28bd-4613-a314-3166234a2fae None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:00.242119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6883a-28bd-4613-a314-3166234a2fae None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:00.242398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6883a-28bd-4613-a314-3166234a2fae None admin] PATH_INFO: `/v3/users/d2c9581524014ff6b6fccab34ed5448f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:00.243626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6883a-28bd-4613-a314-3166234a2fae None admin] RBAC: Authorizing `identity:get_user(user_id=d2c9581524014ff6b6fccab34ed5448f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:00.245085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6883a-28bd-4613-a314-3166234a2fae None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:00.246905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 151/310] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:17:00 2022] GET /identity/v3/users/d2c9581524014ff6b6fccab34ed5448f => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:00.249343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6883a-28bd-4613-a314-3166234a2fae None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:00.276765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ec9bc-16db-44ef-a5d5-c7355a77dbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:00.277426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ec9bc-16db-44ef-a5d5-c7355a77dbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:00.277712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ec9bc-16db-44ef-a5d5-c7355a77dbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:00.278043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ec9bc-16db-44ef-a5d5-c7355a77dbf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:00.278851 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ec9bc-16db-44ef-a5d5-c7355a77dbf0 None admin] RBAC: Adding query filter params (role.id=a889423e7cb541dc99e90f4293116033, user.id=4c7ac0f524d840b683adacc2faf05adb, scope.system=all) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:00.279166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ec9bc-16db-44ef-a5d5-c7355a77dbf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:00.280606 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ec9bc-16db-44ef-a5d5-c7355a77dbf0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:00.293971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 160/311] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jul 21 06:17:00 2022] GET /identity/v3/role_assignments?user.id=4c7ac0f524d840b683adacc2faf05adb&scope.system=all&role.id=a889423e7cb541dc99e90f4293116033 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:00.293971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ec9bc-16db-44ef-a5d5-c7355a77dbf0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:00.383862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6883a-28bd-4613-a314-3166234a2fae None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:00.384502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:00.391182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:00.430905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ec9bc-16db-44ef-a5d5-c7355a77dbf0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:00.435826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60143eae-d19c-4b72-a0d2-61877e8644a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:00.436653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143eae-d19c-4b72-a0d2-61877e8644a0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:00.436943 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143eae-d19c-4b72-a0d2-61877e8644a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:00.437220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143eae-d19c-4b72-a0d2-61877e8644a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:00.452022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60143eae-d19c-4b72-a0d2-61877e8644a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-60143eae-d19c-4b72-a0d2-61877e8644a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:00.462448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:00.463360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:00.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 152/312] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:00.469926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60143eae-d19c-4b72-a0d2-61877e8644a0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:00.483912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b8c3d-30f3-4b7d-9b87-0ff82ef3aed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:00.485351 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b8c3d-30f3-4b7d-9b87-0ff82ef3aed2 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:00.485967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b8c3d-30f3-4b7d-9b87-0ff82ef3aed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:00.485967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b8c3d-30f3-4b7d-9b87-0ff82ef3aed2 None admin] PATH_INFO: `/v3/system/users/d2c9581524014ff6b6fccab34ed5448f/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:00.487582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b8c3d-30f3-4b7d-9b87-0ff82ef3aed2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d2c9581524014ff6b6fccab34ed5448f, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:00.489977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b8c3d-30f3-4b7d-9b87-0ff82ef3aed2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:00.501185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 161/313] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Jul 21 06:17:00 2022] PUT /identity/v3/system/users/d2c9581524014ff6b6fccab34ed5448f/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:17:00.594499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60143eae-d19c-4b72-a0d2-61877e8644a0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:00.634802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537380f5-12a2-4e42-b347-00a55204ea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:00.635524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537380f5-12a2-4e42-b347-00a55204ea71 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:00.635815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537380f5-12a2-4e42-b347-00a55204ea71 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:00.636095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537380f5-12a2-4e42-b347-00a55204ea71 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:00.636881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537380f5-12a2-4e42-b347-00a55204ea71 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:00.637196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537380f5-12a2-4e42-b347-00a55204ea71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:00.638631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537380f5-12a2-4e42-b347-00a55204ea71 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:00.649710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 153/314] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:17:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:00.655341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b8c3d-30f3-4b7d-9b87-0ff82ef3aed2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:00.763081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b8c3d-30f3-4b7d-9b87-0ff82ef3aed2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:00.800799 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357f16a-2339-42a7-8418-8e3c5a2aded4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:00.801684 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f16a-2339-42a7-8418-8e3c5a2aded4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:00.801992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f16a-2339-42a7-8418-8e3c5a2aded4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:00.802271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f16a-2339-42a7-8418-8e3c5a2aded4 None admin] PATH_INFO: `/v3/users/5f636619f809432f9eb1d58743a7f119` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:00.803993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357f16a-2339-42a7-8418-8e3c5a2aded4 None admin] RBAC: Authorizing `identity:get_user(user_id=5f636619f809432f9eb1d58743a7f119)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:00.805502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357f16a-2339-42a7-8418-8e3c5a2aded4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:00.811680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 162/315] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:17:00 2022] GET /identity/v3/users/5f636619f809432f9eb1d58743a7f119 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:00.813078 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537380f5-12a2-4e42-b347-00a55204ea71 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:00.842421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:00.923985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537380f5-12a2-4e42-b347-00a55204ea71 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:00.960097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6fb47-dffd-4ecc-8e1f-221c849bdc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:00.960766 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6fb47-dffd-4ecc-8e1f-221c849bdc1b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:00.961050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6fb47-dffd-4ecc-8e1f-221c849bdc1b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:00.961330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6fb47-dffd-4ecc-8e1f-221c849bdc1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:01.034576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6fb47-dffd-4ecc-8e1f-221c849bdc1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:01.036405 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6fb47-dffd-4ecc-8e1f-221c849bdc1b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:01.066963 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:01.143616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 154/316] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:01.149326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357f16a-2339-42a7-8418-8e3c5a2aded4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:01.153989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:01.257060 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357f16a-2339-42a7-8418-8e3c5a2aded4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:01.296255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408e041-c877-4470-8ecd-2c58dee3e60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:01.297437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408e041-c877-4470-8ecd-2c58dee3e60e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:01.297988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408e041-c877-4470-8ecd-2c58dee3e60e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:01.298467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408e041-c877-4470-8ecd-2c58dee3e60e None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:01.299851 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408e041-c877-4470-8ecd-2c58dee3e60e None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:01.301515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408e041-c877-4470-8ecd-2c58dee3e60e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:01.303939 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 163/317] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:01 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:01.313986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6fb47-dffd-4ecc-8e1f-221c849bdc1b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:01.439364 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6fb47-dffd-4ecc-8e1f-221c849bdc1b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:01.467131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:01.477286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a53ac1-6853-4f68-8530-dfd9233aec63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:01.477968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a53ac1-6853-4f68-8530-dfd9233aec63 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:01.478247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a53ac1-6853-4f68-8530-dfd9233aec63 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:01.478526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a53ac1-6853-4f68-8530-dfd9233aec63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:01.479363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a53ac1-6853-4f68-8530-dfd9233aec63 None admin] RBAC: Adding query filter params (role.id=a889423e7cb541dc99e90f4293116033, scope.project.id=860106fdc2dc43498b8d25ed76d618e8, user.id=5f636619f809432f9eb1d58743a7f119) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:01.479673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a53ac1-6853-4f68-8530-dfd9233aec63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:01.481100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a53ac1-6853-4f68-8530-dfd9233aec63 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:01.494404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 155/318] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:01 2022] GET /identity/v3/role_assignments?user.id=5f636619f809432f9eb1d58743a7f119&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=a889423e7cb541dc99e90f4293116033 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:02.046102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a51dd-7a7d-41de-87d4-8ffec67dbe01 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:02.046477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a51dd-7a7d-41de-87d4-8ffec67dbe01 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:02.046719 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a51dd-7a7d-41de-87d4-8ffec67dbe01 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:02.048026 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 164/319] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:02.056253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072d695-1205-40fc-87f2-f8da3bd9335f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:02.056650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072d695-1205-40fc-87f2-f8da3bd9335f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:02.056801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072d695-1205-40fc-87f2-f8da3bd9335f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:02.073898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c072d695-1205-40fc-87f2-f8da3bd9335f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:02.075104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c072d695-1205-40fc-87f2-f8da3bd9335f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:02.143363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 156/320] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:02.167839 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a51dd-7a7d-41de-87d4-8ffec67dbe01 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:02.241168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:02.276013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a51dd-7a7d-41de-87d4-8ffec67dbe01 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:02.311006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5c014-1329-4bf0-bbc0-f7453a3c523c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:02.311722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c014-1329-4bf0-bbc0-f7453a3c523c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:02.312004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c014-1329-4bf0-bbc0-f7453a3c523c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:02.312281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c014-1329-4bf0-bbc0-f7453a3c523c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:02.313431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5c014-1329-4bf0-bbc0-f7453a3c523c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:02.315111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5c014-1329-4bf0-bbc0-f7453a3c523c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:02.318121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 165/321] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:02.322907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072d695-1205-40fc-87f2-f8da3bd9335f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:02.425482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072d695-1205-40fc-87f2-f8da3bd9335f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:02.445449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:02.461126 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95139331-f0c6-4e9d-b428-0dfed144df7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:02.461805 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95139331-f0c6-4e9d-b428-0dfed144df7a None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:02.462125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95139331-f0c6-4e9d-b428-0dfed144df7a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:02.462409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95139331-f0c6-4e9d-b428-0dfed144df7a None admin] PATH_INFO: `/v3/groups` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:02.463103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95139331-f0c6-4e9d-b428-0dfed144df7a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:02.464552 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95139331-f0c6-4e9d-b428-0dfed144df7a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:02.485643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 157/322] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Jul 21 06:17:02 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:17:02.485865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed52af-1543-4239-92d6-e6412d59cd3e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:02.485865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed52af-1543-4239-92d6-e6412d59cd3e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:02.485865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed52af-1543-4239-92d6-e6412d59cd3e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:02.486311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 166/323] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:02.492313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e607c-504a-45ca-a8d4-2a494daa203b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:02.492625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e607c-504a-45ca-a8d4-2a494daa203b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:02.492872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e607c-504a-45ca-a8d4-2a494daa203b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:02.511284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-427e607c-504a-45ca-a8d4-2a494daa203b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:02.512520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-427e607c-504a-45ca-a8d4-2a494daa203b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:02.598074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 158/324] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:17:02.618117 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed52af-1543-4239-92d6-e6412d59cd3e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:02.628763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cacc92-ef3b-40bb-b348-9a4b9520d1ec None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:02.629065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cacc92-ef3b-40bb-b348-9a4b9520d1ec None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:02.629325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cacc92-ef3b-40bb-b348-9a4b9520d1ec None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:02.630961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 159/325] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:17:02.636255 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af27df-191c-4e13-850a-a15e924c5b46 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:02.636550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af27df-191c-4e13-850a-a15e924c5b46 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:02.636835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af27df-191c-4e13-850a-a15e924c5b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:02.654192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-81af27df-191c-4e13-850a-a15e924c5b46 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:02.655378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-81af27df-191c-4e13-850a-a15e924c5b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:02.737330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 160/326] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:17:02.743511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed52af-1543-4239-92d6-e6412d59cd3e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:02.761638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af27df-191c-4e13-850a-a15e924c5b46 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:02.790285 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a7dc8-39b9-4770-907a-055e51bf794b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:02.791084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a7dc8-39b9-4770-907a-055e51bf794b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:02.791367 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a7dc8-39b9-4770-907a-055e51bf794b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:02.791634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a7dc8-39b9-4770-907a-055e51bf794b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:02.800799 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:02.805080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a7dc8-39b9-4770-907a-055e51bf794b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5f3a7dc8-39b9-4770-907a-055e51bf794b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:02.814279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:02.815169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:02.815952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 167/327] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:02.820945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a7dc8-39b9-4770-907a-055e51bf794b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:02.821422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:02.903125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af27df-191c-4e13-850a-a15e924c5b46 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:02.947809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f60503-420e-46d8-8ec1-98bcca778f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:02.948641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f60503-420e-46d8-8ec1-98bcca778f25 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:02.948923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f60503-420e-46d8-8ec1-98bcca778f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:02.949200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f60503-420e-46d8-8ec1-98bcca778f25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:02.960824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f60503-420e-46d8-8ec1-98bcca778f25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:02.961331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a7dc8-39b9-4770-907a-055e51bf794b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e2f60503-420e-46d8-8ec1-98bcca778f25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:02.968687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:02.969586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:02.970557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 161/328] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:02.979059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f60503-420e-46d8-8ec1-98bcca778f25 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:03.004057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e65fc8-dfee-4c09-b6c8-d662515aa902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:03.004740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e65fc8-dfee-4c09-b6c8-d662515aa902 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:03.005027 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e65fc8-dfee-4c09-b6c8-d662515aa902 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:03.005308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e65fc8-dfee-4c09-b6c8-d662515aa902 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:03.006227 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e65fc8-dfee-4c09-b6c8-d662515aa902 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:03.006660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e65fc8-dfee-4c09-b6c8-d662515aa902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:03.008212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e65fc8-dfee-4c09-b6c8-d662515aa902 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:03.018072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 168/329] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:17:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:03.025387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e65fc8-dfee-4c09-b6c8-d662515aa902 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:03.094539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:03.110538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:03.113429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f60503-420e-46d8-8ec1-98bcca778f25 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:03.143045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e65fc8-dfee-4c09-b6c8-d662515aa902 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:03.151253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aee69-6743-4d76-a6e8-4f0fcf49ec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:03.152278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aee69-6743-4d76-a6e8-4f0fcf49ec8a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:03.152764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aee69-6743-4d76-a6e8-4f0fcf49ec8a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:03.153314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aee69-6743-4d76-a6e8-4f0fcf49ec8a None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:03.154312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aee69-6743-4d76-a6e8-4f0fcf49ec8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:03.154819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aee69-6743-4d76-a6e8-4f0fcf49ec8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:03.156680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aee69-6743-4d76-a6e8-4f0fcf49ec8a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:03.166830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 162/330] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:17:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:03.171906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aee69-6743-4d76-a6e8-4f0fcf49ec8a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:03.187537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d15e3b-af9b-409b-aa7c-4b28fdeb77a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:03.188658 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d15e3b-af9b-409b-aa7c-4b28fdeb77a9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:03.189157 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d15e3b-af9b-409b-aa7c-4b28fdeb77a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:03.189643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d15e3b-af9b-409b-aa7c-4b28fdeb77a9 None admin] PATH_INFO: `/v3/users/4c7ac0f524d840b683adacc2faf05adb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:03.191118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d15e3b-af9b-409b-aa7c-4b28fdeb77a9 None admin] RBAC: Authorizing `identity:get_user(user_id=4c7ac0f524d840b683adacc2faf05adb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:03.192788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d15e3b-af9b-409b-aa7c-4b28fdeb77a9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:03.195323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 169/331] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:17:03 2022] GET /identity/v3/users/4c7ac0f524d840b683adacc2faf05adb => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:03.200928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d15e3b-af9b-409b-aa7c-4b28fdeb77a9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:03.294491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aee69-6743-4d76-a6e8-4f0fcf49ec8a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:03.337335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d15e3b-af9b-409b-aa7c-4b28fdeb77a9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:03.366052 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8ce43-103f-4ad7-a74b-f4c4e03859f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:03.367282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8ce43-103f-4ad7-a74b-f4c4e03859f5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:03.367775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8ce43-103f-4ad7-a74b-f4c4e03859f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:03.369341 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8ce43-103f-4ad7-a74b-f4c4e03859f5 None admin] PATH_INFO: `/v3/users/d2c9581524014ff6b6fccab34ed5448f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:03.371020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8ce43-103f-4ad7-a74b-f4c4e03859f5 None admin] RBAC: Authorizing `identity:get_user(user_id=d2c9581524014ff6b6fccab34ed5448f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:03.372949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8ce43-103f-4ad7-a74b-f4c4e03859f5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:03.376304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:03.376430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 163/332] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:17:03 2022] GET /identity/v3/users/d2c9581524014ff6b6fccab34ed5448f => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:03.383280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8ce43-103f-4ad7-a74b-f4c4e03859f5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:03.394966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d528c-e154-43bd-a66b-3be2d7c49874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:03.397082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d528c-e154-43bd-a66b-3be2d7c49874 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:03.397512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d528c-e154-43bd-a66b-3be2d7c49874 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:03.397923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d528c-e154-43bd-a66b-3be2d7c49874 None admin] PATH_INFO: `/v3/system/users/4c7ac0f524d840b683adacc2faf05adb/roles/a889423e7cb541dc99e90f4293116033` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:03.401599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d528c-e154-43bd-a66b-3be2d7c49874 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4c7ac0f524d840b683adacc2faf05adb, role_id=a889423e7cb541dc99e90f4293116033)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:03.403585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d528c-e154-43bd-a66b-3be2d7c49874 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:03.423261 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 170/333] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Jul 21 06:17:03 2022] PUT /identity/v3/system/users/4c7ac0f524d840b683adacc2faf05adb/roles/a889423e7cb541dc99e90f4293116033 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:17:03.430436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:03.533311 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8ce43-103f-4ad7-a74b-f4c4e03859f5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:03.576176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611d160-3b72-40e3-a507-50154a7ee2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:03.576859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611d160-3b72-40e3-a507-50154a7ee2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:03.577140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611d160-3b72-40e3-a507-50154a7ee2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:03.577418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611d160-3b72-40e3-a507-50154a7ee2d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:03.578307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611d160-3b72-40e3-a507-50154a7ee2d6 None admin] RBAC: Adding query filter params (role.id=d3c75d38368d47d799bf59a0f9c879ea, user.id=d2c9581524014ff6b6fccab34ed5448f, scope.system=all) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:03.578620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611d160-3b72-40e3-a507-50154a7ee2d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:03.580088 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611d160-3b72-40e3-a507-50154a7ee2d6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:03.602368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 164/334] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jul 21 06:17:03 2022] GET /identity/v3/role_assignments?user.id=d2c9581524014ff6b6fccab34ed5448f&scope.system=all&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:05.046988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9484-fd53-4a44-94de-fec000f58580 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:05.047498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9484-fd53-4a44-94de-fec000f58580 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:05.047574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9484-fd53-4a44-94de-fec000f58580 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:05.049034 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 171/335] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:05.055892 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc157ae-5250-4c0f-81a0-10d8f532b6d7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:05.056254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc157ae-5250-4c0f-81a0-10d8f532b6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:05.056407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc157ae-5250-4c0f-81a0-10d8f532b6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:05.082996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3dc157ae-5250-4c0f-81a0-10d8f532b6d7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:05.084691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3dc157ae-5250-4c0f-81a0-10d8f532b6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:05.168600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 165/336] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:17:05.215317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c9484-fd53-4a44-94de-fec000f58580 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:05.363016 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c9484-fd53-4a44-94de-fec000f58580 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:05.415741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f100d-b58d-4cab-a865-42ef6f69c844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:05.416551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f100d-b58d-4cab-a865-42ef6f69c844 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:05.416838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f100d-b58d-4cab-a865-42ef6f69c844 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:05.417120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f100d-b58d-4cab-a865-42ef6f69c844 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:05.438892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f100d-b58d-4cab-a865-42ef6f69c844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1d0f100d-b58d-4cab-a865-42ef6f69c844 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:05.447004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:05.447884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:05.448686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 172/337] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:05.454461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc157ae-5250-4c0f-81a0-10d8f532b6d7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:05.502319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:05.599918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc157ae-5250-4c0f-81a0-10d8f532b6d7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:05.648297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37116c51-a46b-44b2-8670-baeabfd44f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:05.648911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37116c51-a46b-44b2-8670-baeabfd44f6f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:05.649199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37116c51-a46b-44b2-8670-baeabfd44f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:05.649484 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37116c51-a46b-44b2-8670-baeabfd44f6f None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:05.650276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37116c51-a46b-44b2-8670-baeabfd44f6f None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:05.650591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37116c51-a46b-44b2-8670-baeabfd44f6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:05.651984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37116c51-a46b-44b2-8670-baeabfd44f6f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:05.662235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 166/338] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:17:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:05.667981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f100d-b58d-4cab-a865-42ef6f69c844 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:05.734689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca16331-a2ac-4929-9762-40728277581a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:05.735301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca16331-a2ac-4929-9762-40728277581a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:05.735765 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca16331-a2ac-4929-9762-40728277581a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:05.737582 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 167/339] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:05.743188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520bffc-8f8f-422a-87f9-4b6c5ab6058e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:05.743714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520bffc-8f8f-422a-87f9-4b6c5ab6058e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:05.744198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520bffc-8f8f-422a-87f9-4b6c5ab6058e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:05.773645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:05.775000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9520bffc-8f8f-422a-87f9-4b6c5ab6058e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:05.776215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9520bffc-8f8f-422a-87f9-4b6c5ab6058e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:05.891580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f100d-b58d-4cab-a865-42ef6f69c844 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:05.909173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 168/340] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:17:05.941806 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113e2c5-bd39-4eb5-8dc7-64eb8196a833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:05.943313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113e2c5-bd39-4eb5-8dc7-64eb8196a833 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:05.944022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113e2c5-bd39-4eb5-8dc7-64eb8196a833 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:05.944306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113e2c5-bd39-4eb5-8dc7-64eb8196a833 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:05.949494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520bffc-8f8f-422a-87f9-4b6c5ab6058e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:05.969760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:06.077911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113e2c5-bd39-4eb5-8dc7-64eb8196a833 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:06.079436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113e2c5-bd39-4eb5-8dc7-64eb8196a833 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:06.142395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:06.143864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520bffc-8f8f-422a-87f9-4b6c5ab6058e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:06.200933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 173/341] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:06.202229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b85623-81fd-4d56-95a7-07860c8d8367 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:06.202521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b85623-81fd-4d56-95a7-07860c8d8367 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:06.202795 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b85623-81fd-4d56-95a7-07860c8d8367 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:06.204626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 174/342] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:06.211255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386a400-b90c-4575-b708-ddf779c2b215 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:06.212034 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386a400-b90c-4575-b708-ddf779c2b215 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:06.212330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386a400-b90c-4575-b708-ddf779c2b215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:06.231937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3386a400-b90c-4575-b708-ddf779c2b215 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:06.233122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3386a400-b90c-4575-b708-ddf779c2b215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:06.248522 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d81ed8-8bf7-49ee-b003-04750e0bcd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:06.249355 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d81ed8-8bf7-49ee-b003-04750e0bcd9a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:06.249641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d81ed8-8bf7-49ee-b003-04750e0bcd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:06.249921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d81ed8-8bf7-49ee-b003-04750e0bcd9a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:06.265520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d81ed8-8bf7-49ee-b003-04750e0bcd9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c3d81ed8-8bf7-49ee-b003-04750e0bcd9a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:06.276648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:06.277532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:06.278809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 169/343] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:17:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:06.279643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d81ed8-8bf7-49ee-b003-04750e0bcd9a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:06.329221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 175/344] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:17:06.330669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0e225-6b99-4b44-86cb-e87b9a20a925 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:06.334226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0e225-6b99-4b44-86cb-e87b9a20a925 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:06.334545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0e225-6b99-4b44-86cb-e87b9a20a925 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:06.335996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 176/345] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:06.336725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0e225-6b99-4b44-86cb-e87b9a20a925 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:06.398125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:06.440890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d81ed8-8bf7-49ee-b003-04750e0bcd9a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:06.462906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:06.495685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0e225-6b99-4b44-86cb-e87b9a20a925 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:06.496036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f38f81-47aa-4bf3-af3b-85913e6432b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:06.497395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f38f81-47aa-4bf3-af3b-85913e6432b7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:06.497922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f38f81-47aa-4bf3-af3b-85913e6432b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:06.498430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f38f81-47aa-4bf3-af3b-85913e6432b7 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:06.501657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f38f81-47aa-4bf3-af3b-85913e6432b7 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:06.503473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f38f81-47aa-4bf3-af3b-85913e6432b7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:06.505792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 170/346] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:06 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:06.508148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08f15e-9b3f-4032-9c37-d917534d7951 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:06.508652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08f15e-9b3f-4032-9c37-d917534d7951 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:06.509592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08f15e-9b3f-4032-9c37-d917534d7951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:06.531438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cc08f15e-9b3f-4032-9c37-d917534d7951 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:06.532983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc08f15e-9b3f-4032-9c37-d917534d7951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:06.540086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363098c-bbf4-463f-b36f-245a4b8fbf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:06.540086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363098c-bbf4-463f-b36f-245a4b8fbf46 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:06.540086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363098c-bbf4-463f-b36f-245a4b8fbf46 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:06.540086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363098c-bbf4-463f-b36f-245a4b8fbf46 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:06.540716 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363098c-bbf4-463f-b36f-245a4b8fbf46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:06.541027 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363098c-bbf4-463f-b36f-245a4b8fbf46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:06.542999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363098c-bbf4-463f-b36f-245a4b8fbf46 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:06.553006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 177/347] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:06.553979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363098c-bbf4-463f-b36f-245a4b8fbf46 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:06.613409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 171/348] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:06.615392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08f15e-9b3f-4032-9c37-d917534d7951 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:06.667615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363098c-bbf4-463f-b36f-245a4b8fbf46 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:06.704832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4166e9-4add-4794-a377-7b8001dc0f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:06.705644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4166e9-4add-4794-a377-7b8001dc0f15 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:06.705926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4166e9-4add-4794-a377-7b8001dc0f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:06.706230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4166e9-4add-4794-a377-7b8001dc0f15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:06.706622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:06.719291 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4166e9-4add-4794-a377-7b8001dc0f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9c4166e9-4add-4794-a377-7b8001dc0f15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:06.729985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:06.730954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:06.731753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:06.732349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 178/349] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:06.733157 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4166e9-4add-4794-a377-7b8001dc0f15 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:06.737482 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:06.742573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08f15e-9b3f-4032-9c37-d917534d7951 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:06.782957 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d801d9-b08c-46c8-8a56-ea70c8dd488d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:06.783829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d801d9-b08c-46c8-8a56-ea70c8dd488d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:06.784113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d801d9-b08c-46c8-8a56-ea70c8dd488d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:06.784388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d801d9-b08c-46c8-8a56-ea70c8dd488d None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:06.791695 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d801d9-b08c-46c8-8a56-ea70c8dd488d None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:06.793159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d801d9-b08c-46c8-8a56-ea70c8dd488d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:06.796532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 172/350] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:06 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:06.797444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d801d9-b08c-46c8-8a56-ea70c8dd488d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:06.867090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4166e9-4add-4794-a377-7b8001dc0f15 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:06.906184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7927215-0d28-4479-b902-7731da05eed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:06.906850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7927215-0d28-4479-b902-7731da05eed2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:06.907177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7927215-0d28-4479-b902-7731da05eed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:06.907458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7927215-0d28-4479-b902-7731da05eed2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:06.930946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d801d9-b08c-46c8-8a56-ea70c8dd488d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:06.972110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:06.980525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8fbd4-5ef2-4ffa-bc46-df8e3a567924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:06.981348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8fbd4-5ef2-4ffa-bc46-df8e3a567924 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:06.981631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8fbd4-5ef2-4ffa-bc46-df8e3a567924 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:06.981904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8fbd4-5ef2-4ffa-bc46-df8e3a567924 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:06.983238 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7927215-0d28-4479-b902-7731da05eed2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:06.984717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7927215-0d28-4479-b902-7731da05eed2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:06.988389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:06.994984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8fbd4-5ef2-4ffa-bc46-df8e3a567924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-09b8fbd4-5ef2-4ffa-bc46-df8e3a567924 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:07.005195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:07.006435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 06:17:07.007239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:07.008384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 173/351] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:07.008818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8fbd4-5ef2-4ffa-bc46-df8e3a567924 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:07.133678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 179/352] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:07.139456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7927215-0d28-4479-b902-7731da05eed2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:07.142558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8fbd4-5ef2-4ffa-bc46-df8e3a567924 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:07.190687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fd95f-7f26-4119-9b97-4e4f14fe08dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:07.191413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fd95f-7f26-4119-9b97-4e4f14fe08dc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:07.191693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fd95f-7f26-4119-9b97-4e4f14fe08dc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:07.192567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fd95f-7f26-4119-9b97-4e4f14fe08dc None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:07.193834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fd95f-7f26-4119-9b97-4e4f14fe08dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:07.194199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fd95f-7f26-4119-9b97-4e4f14fe08dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:07.196389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fd95f-7f26-4119-9b97-4e4f14fe08dc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:07.207166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 174/353] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:17:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:07.211159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739d6c3-26f2-4b75-a9de-fd8da16a0344 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:07.211454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739d6c3-26f2-4b75-a9de-fd8da16a0344 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:07.211738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739d6c3-26f2-4b75-a9de-fd8da16a0344 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:07.213125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 175/354] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:07.213838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739d6c3-26f2-4b75-a9de-fd8da16a0344 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:07.272447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7927215-0d28-4479-b902-7731da05eed2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:07.276035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:07.296182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:07.326398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa77d04-7a3b-407c-9235-503925e02f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:07.327723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa77d04-7a3b-407c-9235-503925e02f10 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:07.327723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa77d04-7a3b-407c-9235-503925e02f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:07.327723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa77d04-7a3b-407c-9235-503925e02f10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:07.328451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa77d04-7a3b-407c-9235-503925e02f10 None admin] RBAC: Adding query filter params (group.id=c7746236566f49a58fb70f6b2233fb07, role.id=d3c75d38368d47d799bf59a0f9c879ea, scope.project.id=2f47c32296eb4caabf57eb7403220ebf) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:07.328765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa77d04-7a3b-407c-9235-503925e02f10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:07.330244 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa77d04-7a3b-407c-9235-503925e02f10 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:07.336881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 180/355] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 06:17:07 2022] GET /identity/v3/role_assignments?group.id=c7746236566f49a58fb70f6b2233fb07&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:07.337699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa77d04-7a3b-407c-9235-503925e02f10 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:07.356294 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739d6c3-26f2-4b75-a9de-fd8da16a0344 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:07.402472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287fa54-f064-4cd9-98b2-2f21e5a2e0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:07.403176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287fa54-f064-4cd9-98b2-2f21e5a2e0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:07.403462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287fa54-f064-4cd9-98b2-2f21e5a2e0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:07.403862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287fa54-f064-4cd9-98b2-2f21e5a2e0a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:07.405044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287fa54-f064-4cd9-98b2-2f21e5a2e0a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:07.405370 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287fa54-f064-4cd9-98b2-2f21e5a2e0a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:07.406833 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287fa54-f064-4cd9-98b2-2f21e5a2e0a5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:07.417680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 176/356] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:17:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:07.418580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287fa54-f064-4cd9-98b2-2f21e5a2e0a5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:07.469794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa77d04-7a3b-407c-9235-503925e02f10 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:07.516896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf96fab-00ed-45bf-ab5d-c58d844cd8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:07.518162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf96fab-00ed-45bf-ab5d-c58d844cd8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:07.518636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf96fab-00ed-45bf-ab5d-c58d844cd8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:07.519091 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf96fab-00ed-45bf-ab5d-c58d844cd8d3 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:07.520742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf96fab-00ed-45bf-ab5d-c58d844cd8d3 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:07.522635 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf96fab-00ed-45bf-ab5d-c58d844cd8d3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:07.524724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 181/357] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:07 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:07.526389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07324bf7-6f49-4ca9-81f4-6228fa2a8798 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:07.526929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07324bf7-6f49-4ca9-81f4-6228fa2a8798 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:07.527467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07324bf7-6f49-4ca9-81f4-6228fa2a8798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:07.547592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-07324bf7-6f49-4ca9-81f4-6228fa2a8798 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:07.549142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-07324bf7-6f49-4ca9-81f4-6228fa2a8798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:07.564271 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287fa54-f064-4cd9-98b2-2f21e5a2e0a5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:07.583543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:07.592741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:07.616859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fa36c-ab63-45b8-87a3-6c9c14bc2351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:07.617673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fa36c-ab63-45b8-87a3-6c9c14bc2351 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:07.617999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fa36c-ab63-45b8-87a3-6c9c14bc2351 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:07.618282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fa36c-ab63-45b8-87a3-6c9c14bc2351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:07.647153 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 182/358] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:07.648019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07324bf7-6f49-4ca9-81f4-6228fa2a8798 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:07.704216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fa36c-ab63-45b8-87a3-6c9c14bc2351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:07.705735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fa36c-ab63-45b8-87a3-6c9c14bc2351 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:07.773403 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07324bf7-6f49-4ca9-81f4-6228fa2a8798 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:07.826300 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a05d33-9a30-41c7-957e-3f01b7a498b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:07.827848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a05d33-9a30-41c7-957e-3f01b7a498b5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:07.828135 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a05d33-9a30-41c7-957e-3f01b7a498b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:07.828405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a05d33-9a30-41c7-957e-3f01b7a498b5 None admin] PATH_INFO: `/v3/users/4c7ac0f524d840b683adacc2faf05adb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:07.829637 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a05d33-9a30-41c7-957e-3f01b7a498b5 None admin] RBAC: Authorizing `identity:get_user(user_id=4c7ac0f524d840b683adacc2faf05adb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:07.831432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a05d33-9a30-41c7-957e-3f01b7a498b5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:07.833020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 183/359] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:17:07 2022] GET /identity/v3/users/4c7ac0f524d840b683adacc2faf05adb => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:07.834027 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a05d33-9a30-41c7-957e-3f01b7a498b5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:07.836169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 177/360] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:07.837508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805082ff-55eb-4d2a-8923-d7a6d95e5141 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:07.837799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805082ff-55eb-4d2a-8923-d7a6d95e5141 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:07.842494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805082ff-55eb-4d2a-8923-d7a6d95e5141 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:07.844046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 178/361] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:07.844665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805082ff-55eb-4d2a-8923-d7a6d95e5141 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:07.875388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:07.879126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:07.978275 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a05d33-9a30-41c7-957e-3f01b7a498b5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:07.992621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805082ff-55eb-4d2a-8923-d7a6d95e5141 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:08.019269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa550ee8-7bc6-42af-8da7-50c40e3a4a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:08.020935 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa550ee8-7bc6-42af-8da7-50c40e3a4a30 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:08.021223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa550ee8-7bc6-42af-8da7-50c40e3a4a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:08.021503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa550ee8-7bc6-42af-8da7-50c40e3a4a30 None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:08.022853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa550ee8-7bc6-42af-8da7-50c40e3a4a30 None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:08.024407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa550ee8-7bc6-42af-8da7-50c40e3a4a30 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:08.026269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 184/362] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:07 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:08.026945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa550ee8-7bc6-42af-8da7-50c40e3a4a30 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:08.040049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6057bb-dd47-4726-9e9a-d094d5306fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:08.046311 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6057bb-dd47-4726-9e9a-d094d5306fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:08.046742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6057bb-dd47-4726-9e9a-d094d5306fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:08.047260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6057bb-dd47-4726-9e9a-d094d5306fa9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:08.067970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6057bb-dd47-4726-9e9a-d094d5306fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7f6057bb-dd47-4726-9e9a-d094d5306fa9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:08.076508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:08.077435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:08.078344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 179/363] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:17:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:08.079051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331332b7-9864-4315-a19f-bf47dff9bab1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:08.079737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331332b7-9864-4315-a19f-bf47dff9bab1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:08.079737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331332b7-9864-4315-a19f-bf47dff9bab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:08.102446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-331332b7-9864-4315-a19f-bf47dff9bab1 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:08.104028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-331332b7-9864-4315-a19f-bf47dff9bab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:08.182894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa550ee8-7bc6-42af-8da7-50c40e3a4a30 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:08.184191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:08.198229 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 180/364] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:08.198973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331332b7-9864-4315-a19f-bf47dff9bab1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:08.207813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:08.250251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1eb7e-bb95-4c03-891b-1d11a6e2a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:08.250973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1eb7e-bb95-4c03-891b-1d11a6e2a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:08.251173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1eb7e-bb95-4c03-891b-1d11a6e2a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:08.253353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1eb7e-bb95-4c03-891b-1d11a6e2a6e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:08.253353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1eb7e-bb95-4c03-891b-1d11a6e2a6e8 None admin] RBAC: Adding query filter params (role.id=a889423e7cb541dc99e90f4293116033, user.id=4c7ac0f524d840b683adacc2faf05adb, scope.system=all) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:08.253353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1eb7e-bb95-4c03-891b-1d11a6e2a6e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:08.254568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1eb7e-bb95-4c03-891b-1d11a6e2a6e8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:08.267451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 185/365] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jul 21 06:17:08 2022] GET /identity/v3/role_assignments?user.id=4c7ac0f524d840b683adacc2faf05adb&scope.system=all&role.id=a889423e7cb541dc99e90f4293116033 => generated 510 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:08.268276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1eb7e-bb95-4c03-891b-1d11a6e2a6e8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:08.340638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331332b7-9864-4315-a19f-bf47dff9bab1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:08.387784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57b953-5196-42d3-82de-b6f84092bd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:08.388678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57b953-5196-42d3-82de-b6f84092bd12 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:08.388969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57b953-5196-42d3-82de-b6f84092bd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:08.389252 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57b953-5196-42d3-82de-b6f84092bd12 None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:08.390697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57b953-5196-42d3-82de-b6f84092bd12 None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:08.392278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57b953-5196-42d3-82de-b6f84092bd12 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:08.394076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 181/366] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:08 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:08.399556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57b953-5196-42d3-82de-b6f84092bd12 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:08.419353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1eb7e-bb95-4c03-891b-1d11a6e2a6e8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:08.470981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c1773-000f-416b-89e4-8070eded9c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:08.471951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c1773-000f-416b-89e4-8070eded9c39 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:08.472514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c1773-000f-416b-89e4-8070eded9c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:08.473545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c1773-000f-416b-89e4-8070eded9c39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:08.475185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c1773-000f-416b-89e4-8070eded9c39 None admin] RBAC: Adding query filter params (group.id=c7746236566f49a58fb70f6b2233fb07, role.id=d534ab3d0a904f0babeb90d2679d1b86, scope.project.id=2f47c32296eb4caabf57eb7403220ebf) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:08.475839 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c1773-000f-416b-89e4-8070eded9c39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:08.477874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c1773-000f-416b-89e4-8070eded9c39 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:08.486892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 186/367] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 06:17:08 2022] GET /identity/v3/role_assignments?group.id=c7746236566f49a58fb70f6b2233fb07&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=d534ab3d0a904f0babeb90d2679d1b86 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:08.488705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c1773-000f-416b-89e4-8070eded9c39 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:08.510319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:08.530391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:08.561277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57b953-5196-42d3-82de-b6f84092bd12 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:08.617837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8a3a8-b916-4ca0-9d95-75349c327c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:08.618889 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8a3a8-b916-4ca0-9d95-75349c327c49 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:08.619376 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8a3a8-b916-4ca0-9d95-75349c327c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:08.619659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8a3a8-b916-4ca0-9d95-75349c327c49 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:08.620438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8a3a8-b916-4ca0-9d95-75349c327c49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:08.620752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8a3a8-b916-4ca0-9d95-75349c327c49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:08.622203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8a3a8-b916-4ca0-9d95-75349c327c49 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:08.636955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 182/368] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:08.637931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8a3a8-b916-4ca0-9d95-75349c327c49 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:08.639512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c1773-000f-416b-89e4-8070eded9c39 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:08.687551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a9ef2-5f87-4654-9deb-a884879416f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:08.688713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a9ef2-5f87-4654-9deb-a884879416f1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:08.689197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a9ef2-5f87-4654-9deb-a884879416f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:08.690660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a9ef2-5f87-4654-9deb-a884879416f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:08.707070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a9ef2-5f87-4654-9deb-a884879416f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bc2a9ef2-5f87-4654-9deb-a884879416f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:08.718865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:08.719789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:08.720577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:08.724249 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 187/369] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:08.725298 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a9ef2-5f87-4654-9deb-a884879416f1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:08.786292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8a3a8-b916-4ca0-9d95-75349c327c49 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:08.829260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9e005-704d-45f9-bfdc-88cb8bdc85c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:08.830559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9e005-704d-45f9-bfdc-88cb8bdc85c2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:08.831058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9e005-704d-45f9-bfdc-88cb8bdc85c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:08.831496 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9e005-704d-45f9-bfdc-88cb8bdc85c2 None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:08.833630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9e005-704d-45f9-bfdc-88cb8bdc85c2 None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:08.835372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9e005-704d-45f9-bfdc-88cb8bdc85c2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:08.836886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:08.838444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 183/370] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:08 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:08.839397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9e005-704d-45f9-bfdc-88cb8bdc85c2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:08.845126 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:08.859425 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a9ef2-5f87-4654-9deb-a884879416f1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:08.912965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f5291-e69a-47e6-be84-4a47bd9e772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:08.913657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f5291-e69a-47e6-be84-4a47bd9e772c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:08.914059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f5291-e69a-47e6-be84-4a47bd9e772c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:08.914412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f5291-e69a-47e6-be84-4a47bd9e772c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:08.954810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9e005-704d-45f9-bfdc-88cb8bdc85c2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:08.997022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03097d-2720-472d-9b85-55c91e0045f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:08.998012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03097d-2720-472d-9b85-55c91e0045f2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:08.998498 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03097d-2720-472d-9b85-55c91e0045f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:08.998954 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03097d-2720-472d-9b85-55c91e0045f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:08.999907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03097d-2720-472d-9b85-55c91e0045f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:09.000300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03097d-2720-472d-9b85-55c91e0045f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:09.001744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03097d-2720-472d-9b85-55c91e0045f2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:09.011340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 184/371] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:09.012230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03097d-2720-472d-9b85-55c91e0045f2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:09.016584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f5291-e69a-47e6-be84-4a47bd9e772c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:09.018324 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f5291-e69a-47e6-be84-4a47bd9e772c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:09.111915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:09.145571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:09.164690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 188/372] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:09.164867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03097d-2720-472d-9b85-55c91e0045f2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:09.165839 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f5291-e69a-47e6-be84-4a47bd9e772c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:09.210841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c156703-a06a-4226-9f4f-e6d0962ad1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:09.211519 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c156703-a06a-4226-9f4f-e6d0962ad1bd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:09.211808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c156703-a06a-4226-9f4f-e6d0962ad1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:09.212089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c156703-a06a-4226-9f4f-e6d0962ad1bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:09.212940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c156703-a06a-4226-9f4f-e6d0962ad1bd None admin] RBAC: Adding query filter params (group.id=c7746236566f49a58fb70f6b2233fb07, role.id=d3c75d38368d47d799bf59a0f9c879ea, scope.project.id=860106fdc2dc43498b8d25ed76d618e8) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:09.213258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c156703-a06a-4226-9f4f-e6d0962ad1bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:09.215105 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c156703-a06a-4226-9f4f-e6d0962ad1bd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:09.223237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 185/373] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 06:17:09 2022] GET /identity/v3/role_assignments?group.id=c7746236566f49a58fb70f6b2233fb07&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:09.224149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c156703-a06a-4226-9f4f-e6d0962ad1bd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:09.294159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f5291-e69a-47e6-be84-4a47bd9e772c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:09.373014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83313a19-6efc-4904-b246-9e127428b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:09.374084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83313a19-6efc-4904-b246-9e127428b003 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:09.374577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83313a19-6efc-4904-b246-9e127428b003 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:09.375076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83313a19-6efc-4904-b246-9e127428b003 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:09.415660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c156703-a06a-4226-9f4f-e6d0962ad1bd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:09.471505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:09.477617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:09.489698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8153f3b-d61e-4dd9-97d0-87e343db91ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:09.490674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8153f3b-d61e-4dd9-97d0-87e343db91ca None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:09.490960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8153f3b-d61e-4dd9-97d0-87e343db91ca None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:09.491237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8153f3b-d61e-4dd9-97d0-87e343db91ca None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:09.494996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8153f3b-d61e-4dd9-97d0-87e343db91ca None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:09.496238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8153f3b-d61e-4dd9-97d0-87e343db91ca None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:09.497421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83313a19-6efc-4904-b246-9e127428b003 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:09.499123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83313a19-6efc-4904-b246-9e127428b003 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:09.502063 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 186/374] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:09 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:09.503207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8153f3b-d61e-4dd9-97d0-87e343db91ca None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:09.610359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 189/375] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:09.617258 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83313a19-6efc-4904-b246-9e127428b003 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:09.662364 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8153f3b-d61e-4dd9-97d0-87e343db91ca None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:09.698583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7b527-96c3-47e4-b341-141262f58ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:09.699396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7b527-96c3-47e4-b341-141262f58ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:09.699681 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7b527-96c3-47e4-b341-141262f58ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:09.699964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7b527-96c3-47e4-b341-141262f58ed7 None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:09.701238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7b527-96c3-47e4-b341-141262f58ed7 None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:09.703824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7b527-96c3-47e4-b341-141262f58ed7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:09.705795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 187/376] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:09 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:09.712131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7b527-96c3-47e4-b341-141262f58ed7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:09.729793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:09.750142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83313a19-6efc-4904-b246-9e127428b003 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:09.787907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:09.789316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94764aa3-c82b-4147-8241-1b230ff3fbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:09.790640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94764aa3-c82b-4147-8241-1b230ff3fbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:09.791128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94764aa3-c82b-4147-8241-1b230ff3fbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:09.791604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94764aa3-c82b-4147-8241-1b230ff3fbd5 None admin] PATH_INFO: `/v3/projects/c098b85eefc14c3cac1c96ebb357b960` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:09.793043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94764aa3-c82b-4147-8241-1b230ff3fbd5 None admin] RBAC: Authorizing `identity:get_project(project_id=c098b85eefc14c3cac1c96ebb357b960)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:09.794800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94764aa3-c82b-4147-8241-1b230ff3fbd5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:09.797183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 190/377] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:09 2022] GET /identity/v3/projects/c098b85eefc14c3cac1c96ebb357b960 => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:09.798673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c2e10-f9f3-469d-9b92-87263f30679a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:09.799166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c2e10-f9f3-469d-9b92-87263f30679a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:09.799660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c2e10-f9f3-469d-9b92-87263f30679a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:09.801417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 191/378] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:09.802410 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c2e10-f9f3-469d-9b92-87263f30679a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:09.827164 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7b527-96c3-47e4-b341-141262f58ed7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:09.865200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada382e-b4f8-4c8b-88fc-275d241ec7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:09.869289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada382e-b4f8-4c8b-88fc-275d241ec7de None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:09.869831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada382e-b4f8-4c8b-88fc-275d241ec7de None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:09.870336 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada382e-b4f8-4c8b-88fc-275d241ec7de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:09.871391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada382e-b4f8-4c8b-88fc-275d241ec7de None admin] RBAC: Adding query filter params (group.id=c7746236566f49a58fb70f6b2233fb07, role.id=d534ab3d0a904f0babeb90d2679d1b86, scope.project.id=860106fdc2dc43498b8d25ed76d618e8) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:09.871925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada382e-b4f8-4c8b-88fc-275d241ec7de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:09.873585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada382e-b4f8-4c8b-88fc-275d241ec7de None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:09.880260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 188/379] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 06:17:09 2022] GET /identity/v3/role_assignments?group.id=c7746236566f49a58fb70f6b2233fb07&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=d534ab3d0a904f0babeb90d2679d1b86 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:09.881815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd2e89-86d3-4a2d-8c31-bd95c8035d23 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:09.882347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd2e89-86d3-4a2d-8c31-bd95c8035d23 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:09.882844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd2e89-86d3-4a2d-8c31-bd95c8035d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:09.907905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c9fd2e89-86d3-4a2d-8c31-bd95c8035d23 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:09.909518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c9fd2e89-86d3-4a2d-8c31-bd95c8035d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:09.962509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c2e10-f9f3-469d-9b92-87263f30679a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:09.994442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 189/380] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:10.024909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd2e89-86d3-4a2d-8c31-bd95c8035d23 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:10.043828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94dcbd-dbca-4a29-b027-3cf3e351cecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:10.043828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94dcbd-dbca-4a29-b027-3cf3e351cecc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:10.050444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94dcbd-dbca-4a29-b027-3cf3e351cecc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:10.050781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94dcbd-dbca-4a29-b027-3cf3e351cecc None admin] PATH_INFO: `/v3/groups/29108e6c5c8c4e09b24a9d3d131f3e75` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:10.076580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94dcbd-dbca-4a29-b027-3cf3e351cecc None admin] RBAC: Authorizing `identity:get_group(group_id=29108e6c5c8c4e09b24a9d3d131f3e75)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:10.078092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94dcbd-dbca-4a29-b027-3cf3e351cecc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:10.079873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 192/381] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:09 2022] GET /identity/v3/groups/29108e6c5c8c4e09b24a9d3d131f3e75 => generated 229 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:10.086939 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94dcbd-dbca-4a29-b027-3cf3e351cecc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:10.110575 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:10.116985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:10.184580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd2e89-86d3-4a2d-8c31-bd95c8035d23 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:10.251537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94dcbd-dbca-4a29-b027-3cf3e351cecc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:10.260545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57170a-cee8-4f7b-859c-22b1347f18cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:10.261365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57170a-cee8-4f7b-859c-22b1347f18cc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:10.261647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57170a-cee8-4f7b-859c-22b1347f18cc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:10.261921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57170a-cee8-4f7b-859c-22b1347f18cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:10.278502 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57170a-cee8-4f7b-859c-22b1347f18cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-cc57170a-cee8-4f7b-859c-22b1347f18cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:10.287024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:10.287905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:10.288674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 190/382] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:10.300406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57170a-cee8-4f7b-859c-22b1347f18cc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:10.325371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabcfae-51f1-4cbf-894e-44f045f27e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:10.326075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabcfae-51f1-4cbf-894e-44f045f27e6c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:10.326359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabcfae-51f1-4cbf-894e-44f045f27e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:10.326633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabcfae-51f1-4cbf-894e-44f045f27e6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:10.327479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabcfae-51f1-4cbf-894e-44f045f27e6c None admin] RBAC: Adding query filter params (group.id=29108e6c5c8c4e09b24a9d3d131f3e75, role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=c098b85eefc14c3cac1c96ebb357b960) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:10.327793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabcfae-51f1-4cbf-894e-44f045f27e6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:10.329263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabcfae-51f1-4cbf-894e-44f045f27e6c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:10.346230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 193/383] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 06:17:10 2022] GET /identity/v3/role_assignments?group.id=29108e6c5c8c4e09b24a9d3d131f3e75&scope.project.id=c098b85eefc14c3cac1c96ebb357b960&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:10.433534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57170a-cee8-4f7b-859c-22b1347f18cc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:10.456801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:10.475023 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10930528-bf6c-4fbc-9b14-247cf1ec3507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:10.475719 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10930528-bf6c-4fbc-9b14-247cf1ec3507 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:10.476002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10930528-bf6c-4fbc-9b14-247cf1ec3507 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:10.476278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10930528-bf6c-4fbc-9b14-247cf1ec3507 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:10.477057 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10930528-bf6c-4fbc-9b14-247cf1ec3507 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:10.477366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10930528-bf6c-4fbc-9b14-247cf1ec3507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:10.478832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10930528-bf6c-4fbc-9b14-247cf1ec3507 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:10.488988 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 191/384] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:17:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:10.494976 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabcfae-51f1-4cbf-894e-44f045f27e6c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:10.600139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:10.622794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabcfae-51f1-4cbf-894e-44f045f27e6c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:10.662794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aa1e8-5937-4c21-9f4d-5616d3a0a631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:10.663686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aa1e8-5937-4c21-9f4d-5616d3a0a631 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:10.663973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aa1e8-5937-4c21-9f4d-5616d3a0a631 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:10.664252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aa1e8-5937-4c21-9f4d-5616d3a0a631 None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:10.665467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894aa1e8-5937-4c21-9f4d-5616d3a0a631 None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:10.666957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894aa1e8-5937-4c21-9f4d-5616d3a0a631 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:10.669265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 194/385] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:10 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:10.674653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10930528-bf6c-4fbc-9b14-247cf1ec3507 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:10.788576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10930528-bf6c-4fbc-9b14-247cf1ec3507 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:10.825050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be6248-4d23-421b-b7c6-38784dd12136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:10.825921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be6248-4d23-421b-b7c6-38784dd12136 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:10.826232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be6248-4d23-421b-b7c6-38784dd12136 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:10.826506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be6248-4d23-421b-b7c6-38784dd12136 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:10.827722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be6248-4d23-421b-b7c6-38784dd12136 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:10.829162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be6248-4d23-421b-b7c6-38784dd12136 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:10.831239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 192/386] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:10 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:10.836954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aa1e8-5937-4c21-9f4d-5616d3a0a631 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:10.954483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aa1e8-5937-4c21-9f4d-5616d3a0a631 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:10.995543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5db76e-3f03-46ce-bb2a-88afcb2c205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:10.996435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5db76e-3f03-46ce-bb2a-88afcb2c205c None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:10.996723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5db76e-3f03-46ce-bb2a-88afcb2c205c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:10.997002 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5db76e-3f03-46ce-bb2a-88afcb2c205c None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf/groups/c7746236566f49a58fb70f6b2233fb07/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:10.999006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5db76e-3f03-46ce-bb2a-88afcb2c205c None admin] RBAC: Authorizing `identity:create_grant(project_id=2f47c32296eb4caabf57eb7403220ebf, group_id=c7746236566f49a58fb70f6b2233fb07, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:11.000489 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5db76e-3f03-46ce-bb2a-88afcb2c205c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:11.027231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 195/387] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jul 21 06:17:10 2022] PUT /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf/groups/c7746236566f49a58fb70f6b2233fb07/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:17:11.388653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f0b76-f38e-47d6-b473-250b3a44d7b8 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:11.388653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f0b76-f38e-47d6-b473-250b3a44d7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:11.388653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f0b76-f38e-47d6-b473-250b3a44d7b8 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:11.389812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 193/388] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:11.393787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69dad5-64af-4824-a149-44571e3a013a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:11.394821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69dad5-64af-4824-a149-44571e3a013a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:11.394821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69dad5-64af-4824-a149-44571e3a013a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:11.412819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-eb69dad5-64af-4824-a149-44571e3a013a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:11.414035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eb69dad5-64af-4824-a149-44571e3a013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:11.416173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:11.489793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 196/389] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:11.512516 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f0b76-f38e-47d6-b473-250b3a44d7b8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:11.567826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:11.623671 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f0b76-f38e-47d6-b473-250b3a44d7b8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:11.664149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6dfad-d58a-4c2b-a5de-bd66db2412cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:11.664917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6dfad-d58a-4c2b-a5de-bd66db2412cc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:11.665204 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6dfad-d58a-4c2b-a5de-bd66db2412cc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:11.665479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6dfad-d58a-4c2b-a5de-bd66db2412cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:11.681258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6dfad-d58a-4c2b-a5de-bd66db2412cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-37c6dfad-d58a-4c2b-a5de-bd66db2412cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:11.689589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:11.690514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:11.691589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 194/390] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:17:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:11.697308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69dad5-64af-4824-a149-44571e3a013a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:11.839370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69dad5-64af-4824-a149-44571e3a013a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:11.888242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b93cc1-87b3-4e70-8dd5-aa50bca14d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:11.888938 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b93cc1-87b3-4e70-8dd5-aa50bca14d97 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:11.889224 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b93cc1-87b3-4e70-8dd5-aa50bca14d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:11.889505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b93cc1-87b3-4e70-8dd5-aa50bca14d97 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:11.890302 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b93cc1-87b3-4e70-8dd5-aa50bca14d97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:11.890615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b93cc1-87b3-4e70-8dd5-aa50bca14d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:11.892017 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b93cc1-87b3-4e70-8dd5-aa50bca14d97 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:11.898820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:11.903522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 197/391] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:11.911159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6dfad-d58a-4c2b-a5de-bd66db2412cc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:12.035645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e74220-0882-4e86-9d36-e3348eaec349 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:12.036034 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e74220-0882-4e86-9d36-e3348eaec349 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:12.036399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e74220-0882-4e86-9d36-e3348eaec349 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:12.038195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 198/392] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:12.043753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c34fd-e434-4f41-85bd-919903e389a6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:12.043753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c34fd-e434-4f41-85bd-919903e389a6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:12.043753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c34fd-e434-4f41-85bd-919903e389a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:12.077391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a15c34fd-e434-4f41-85bd-919903e389a6 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:12.079415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a15c34fd-e434-4f41-85bd-919903e389a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:12.082198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6dfad-d58a-4c2b-a5de-bd66db2412cc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:12.101521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:12.141073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b78a0-85bb-40d9-aa0f-f14dc1e32d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:12.141965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b78a0-85bb-40d9-aa0f-f14dc1e32d7f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:12.142259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b78a0-85bb-40d9-aa0f-f14dc1e32d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:12.142538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b78a0-85bb-40d9-aa0f-f14dc1e32d7f None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:12.145274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b78a0-85bb-40d9-aa0f-f14dc1e32d7f None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:12.146755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b78a0-85bb-40d9-aa0f-f14dc1e32d7f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:12.150054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 195/393] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:11 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:12.157404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b78a0-85bb-40d9-aa0f-f14dc1e32d7f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:12.207854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 199/394] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:12.229385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c34fd-e434-4f41-85bd-919903e389a6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:12.323530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b78a0-85bb-40d9-aa0f-f14dc1e32d7f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:12.370275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a9ba8-7305-4726-babe-8a7fa693393a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:12.371163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9ba8-7305-4726-babe-8a7fa693393a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:12.371451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9ba8-7305-4726-babe-8a7fa693393a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:12.371725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9ba8-7305-4726-babe-8a7fa693393a None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:12.373098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a9ba8-7305-4726-babe-8a7fa693393a None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:12.374572 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a9ba8-7305-4726-babe-8a7fa693393a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:12.376443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 196/395] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:12 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:12.383626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a9ba8-7305-4726-babe-8a7fa693393a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:12.422584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c34fd-e434-4f41-85bd-919903e389a6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:12.449342 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:12.460686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:12.469227 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cbef5-39ef-45b0-a639-b361bfdd0dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:12.470750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cbef5-39ef-45b0-a639-b361bfdd0dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:12.471544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cbef5-39ef-45b0-a639-b361bfdd0dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:12.471989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cbef5-39ef-45b0-a639-b361bfdd0dc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:12.485776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cbef5-39ef-45b0-a639-b361bfdd0dc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f68cbef5-39ef-45b0-a639-b361bfdd0dc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:12.496469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:12.497516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:12.498518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:12.501511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 200/396] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:12.504051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc868-38f3-467f-b6f6-389fd0df3c18 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:12.505073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc868-38f3-467f-b6f6-389fd0df3c18 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:12.505626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc868-38f3-467f-b6f6-389fd0df3c18 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:12.507756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 201/397] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:12.508823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcc868-38f3-467f-b6f6-389fd0df3c18 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:12.517669 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a9ba8-7305-4726-babe-8a7fa693393a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:12.561063 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e6902-7474-4b0d-bc2e-14ec2dfa88d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:12.562727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e6902-7474-4b0d-bc2e-14ec2dfa88d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:12.563508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e6902-7474-4b0d-bc2e-14ec2dfa88d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:12.563963 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e6902-7474-4b0d-bc2e-14ec2dfa88d9 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf/groups/c7746236566f49a58fb70f6b2233fb07/roles/d534ab3d0a904f0babeb90d2679d1b86` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:12.567613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e6902-7474-4b0d-bc2e-14ec2dfa88d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f47c32296eb4caabf57eb7403220ebf, group_id=c7746236566f49a58fb70f6b2233fb07, role_id=d534ab3d0a904f0babeb90d2679d1b86)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:12.569665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e6902-7474-4b0d-bc2e-14ec2dfa88d9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:12.598318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 197/398] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jul 21 06:17:12 2022] PUT /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf/groups/c7746236566f49a58fb70f6b2233fb07/roles/d534ab3d0a904f0babeb90d2679d1b86 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:17:12.598318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fc335-d30a-477e-99f2-68e4ae8d422f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:12.598318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fc335-d30a-477e-99f2-68e4ae8d422f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:12.598318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fc335-d30a-477e-99f2-68e4ae8d422f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:12.635197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f87fc335-d30a-477e-99f2-68e4ae8d422f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:12.636409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f87fc335-d30a-477e-99f2-68e4ae8d422f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:12.669969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcc868-38f3-467f-b6f6-389fd0df3c18 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:12.710908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f94642-fdce-4f66-9e2b-2ad3f895d1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:12.711848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f94642-fdce-4f66-9e2b-2ad3f895d1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:12.712309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f94642-fdce-4f66-9e2b-2ad3f895d1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:12.712746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f94642-fdce-4f66-9e2b-2ad3f895d1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:12.713804 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f94642-fdce-4f66-9e2b-2ad3f895d1d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:12.714297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f94642-fdce-4f66-9e2b-2ad3f895d1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:12.715998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f94642-fdce-4f66-9e2b-2ad3f895d1d8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:12.727514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 202/399] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:17:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:12.733213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f94642-fdce-4f66-9e2b-2ad3f895d1d8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:12.809395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 198/400] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:12.818884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:12.845360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fc335-d30a-477e-99f2-68e4ae8d422f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:12.880862 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f94642-fdce-4f66-9e2b-2ad3f895d1d8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:12.890351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:12.937386 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11afdb-b8c0-40af-85e3-63cb6bb874f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:12.938295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11afdb-b8c0-40af-85e3-63cb6bb874f4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:12.938581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11afdb-b8c0-40af-85e3-63cb6bb874f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:12.938860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11afdb-b8c0-40af-85e3-63cb6bb874f4 None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:12.940140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11afdb-b8c0-40af-85e3-63cb6bb874f4 None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:12.942207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11afdb-b8c0-40af-85e3-63cb6bb874f4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:12.944220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 203/401] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:12 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:12.945618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0b46-ed9f-4833-a913-5626d35eef0b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:12.945910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0b46-ed9f-4833-a913-5626d35eef0b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:12.946208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0b46-ed9f-4833-a913-5626d35eef0b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:12.947558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 204/402] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:12.949353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f0b46-ed9f-4833-a913-5626d35eef0b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:12.976738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fc335-d30a-477e-99f2-68e4ae8d422f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:13.015100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef4451-2ff6-45d4-9d6d-5e1979ce9420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:13.016022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef4451-2ff6-45d4-9d6d-5e1979ce9420 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:13.016410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef4451-2ff6-45d4-9d6d-5e1979ce9420 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:13.016698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef4451-2ff6-45d4-9d6d-5e1979ce9420 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:13.027936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef4451-2ff6-45d4-9d6d-5e1979ce9420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-78ef4451-2ff6-45d4-9d6d-5e1979ce9420 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:13.037029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:13.037924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:13.039081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 199/403] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:17:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:13.045754 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200515d-ec22-475b-9594-fa4c493564f9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:13.045754 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200515d-ec22-475b-9594-fa4c493564f9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:13.045754 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200515d-ec22-475b-9594-fa4c493564f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:13.059739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f0b46-ed9f-4833-a913-5626d35eef0b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:13.063989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6200515d-ec22-475b-9594-fa4c493564f9 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:13.065181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6200515d-ec22-475b-9594-fa4c493564f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:13.082374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:13.098846 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908142c-9fec-4ad9-adb2-14f79b58b35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:13.100003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908142c-9fec-4ad9-adb2-14f79b58b35e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:13.100323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908142c-9fec-4ad9-adb2-14f79b58b35e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:13.100615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908142c-9fec-4ad9-adb2-14f79b58b35e None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:13.102608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908142c-9fec-4ad9-adb2-14f79b58b35e None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:13.104382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908142c-9fec-4ad9-adb2-14f79b58b35e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:13.106445 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 205/404] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:12 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:13.107517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908142c-9fec-4ad9-adb2-14f79b58b35e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:13.145766 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 200/405] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:13.146984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200515d-ec22-475b-9594-fa4c493564f9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:13.192953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:13.243967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908142c-9fec-4ad9-adb2-14f79b58b35e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:13.279444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104aee-cef6-4498-945a-e59cc01c2433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:13.280207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104aee-cef6-4498-945a-e59cc01c2433 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:13.280559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104aee-cef6-4498-945a-e59cc01c2433 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:13.280907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104aee-cef6-4498-945a-e59cc01c2433 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:13.281723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f104aee-cef6-4498-945a-e59cc01c2433 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:13.281961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200515d-ec22-475b-9594-fa4c493564f9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:13.282184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f104aee-cef6-4498-945a-e59cc01c2433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:13.283635 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f104aee-cef6-4498-945a-e59cc01c2433 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:13.292892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 206/406] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:13.293864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104aee-cef6-4498-945a-e59cc01c2433 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:13.321063 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a7f12-6e82-4412-8848-603a54ca69d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:13.322037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a7f12-6e82-4412-8848-603a54ca69d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:13.322291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a7f12-6e82-4412-8848-603a54ca69d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:13.322575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a7f12-6e82-4412-8848-603a54ca69d8 None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8/groups/c7746236566f49a58fb70f6b2233fb07/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:13.324736 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a7f12-6e82-4412-8848-603a54ca69d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=860106fdc2dc43498b8d25ed76d618e8, group_id=c7746236566f49a58fb70f6b2233fb07, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:13.326375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a7f12-6e82-4412-8848-603a54ca69d8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:13.342464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 201/407] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jul 21 06:17:13 2022] PUT /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8/groups/c7746236566f49a58fb70f6b2233fb07/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:17:13.343371 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a7f12-6e82-4412-8848-603a54ca69d8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:13.350158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:13.407651 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104aee-cef6-4498-945a-e59cc01c2433 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:13.444501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c698ecf-1489-4775-ae69-dbcd152d5a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:13.445328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c698ecf-1489-4775-ae69-dbcd152d5a9e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:13.445614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c698ecf-1489-4775-ae69-dbcd152d5a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:13.445892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c698ecf-1489-4775-ae69-dbcd152d5a9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:13.449487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:13.461110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c698ecf-1489-4775-ae69-dbcd152d5a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:13.470224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a7f12-6e82-4412-8848-603a54ca69d8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9c698ecf-1489-4775-ae69-dbcd152d5a9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:13.471019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:13.472875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:13.473690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 207/408] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:13.481028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c698ecf-1489-4775-ae69-dbcd152d5a9e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:13.548681 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11fa2e-5a7a-44de-aa0a-d8e7e269b577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:13.550623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11fa2e-5a7a-44de-aa0a-d8e7e269b577 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:13.551250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11fa2e-5a7a-44de-aa0a-d8e7e269b577 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:13.552459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11fa2e-5a7a-44de-aa0a-d8e7e269b577 None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:13.555264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11fa2e-5a7a-44de-aa0a-d8e7e269b577 None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:13.557966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11fa2e-5a7a-44de-aa0a-d8e7e269b577 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:13.559825 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 202/409] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:13 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:13.568553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11fa2e-5a7a-44de-aa0a-d8e7e269b577 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:13.641136 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c698ecf-1489-4775-ae69-dbcd152d5a9e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:13.654885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:13.688390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a713ea8-65c0-4243-8760-f2008b445d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:13.688993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a713ea8-65c0-4243-8760-f2008b445d69 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:13.689277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a713ea8-65c0-4243-8760-f2008b445d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:13.689553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a713ea8-65c0-4243-8760-f2008b445d69 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:13.690373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a713ea8-65c0-4243-8760-f2008b445d69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:13.691312 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a713ea8-65c0-4243-8760-f2008b445d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:13.692698 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a713ea8-65c0-4243-8760-f2008b445d69 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:13.715054 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 208/410] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:13.723730 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a713ea8-65c0-4243-8760-f2008b445d69 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:13.724089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11fa2e-5a7a-44de-aa0a-d8e7e269b577 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:13.782077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91f233-b990-427a-8c6b-d96ea267a02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:13.782447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91f233-b990-427a-8c6b-d96ea267a02e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:13.782749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91f233-b990-427a-8c6b-d96ea267a02e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:13.783031 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91f233-b990-427a-8c6b-d96ea267a02e None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:13.788624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91f233-b990-427a-8c6b-d96ea267a02e None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:13.788624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91f233-b990-427a-8c6b-d96ea267a02e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:13.788897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 203/411] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:13 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:13.806178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91f233-b990-427a-8c6b-d96ea267a02e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:13.862275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:13.864841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a713ea8-65c0-4243-8760-f2008b445d69 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:13.900896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39080fe9-6302-4fdd-8411-68a1f1cee600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:13.902748 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39080fe9-6302-4fdd-8411-68a1f1cee600 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:13.903224 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39080fe9-6302-4fdd-8411-68a1f1cee600 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:13.903936 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39080fe9-6302-4fdd-8411-68a1f1cee600 None admin] PATH_INFO: `/v3/groups/29108e6c5c8c4e09b24a9d3d131f3e75` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:13.905307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39080fe9-6302-4fdd-8411-68a1f1cee600 None admin] RBAC: Authorizing `identity:get_group(group_id=29108e6c5c8c4e09b24a9d3d131f3e75)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:13.906998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39080fe9-6302-4fdd-8411-68a1f1cee600 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:13.909196 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 209/412] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:13 2022] GET /identity/v3/groups/29108e6c5c8c4e09b24a9d3d131f3e75 => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:13.920424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39080fe9-6302-4fdd-8411-68a1f1cee600 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:13.971826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:13.982215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91f233-b990-427a-8c6b-d96ea267a02e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:14.026312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c307b5e-63d8-4d86-9efa-003cef1cbf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:14.028030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c307b5e-63d8-4d86-9efa-003cef1cbf1d None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:14.028821 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c307b5e-63d8-4d86-9efa-003cef1cbf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:14.029260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c307b5e-63d8-4d86-9efa-003cef1cbf1d None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8/groups/c7746236566f49a58fb70f6b2233fb07/roles/d534ab3d0a904f0babeb90d2679d1b86` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:14.032159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c307b5e-63d8-4d86-9efa-003cef1cbf1d None admin] RBAC: Authorizing `identity:create_grant(project_id=860106fdc2dc43498b8d25ed76d618e8, group_id=c7746236566f49a58fb70f6b2233fb07, role_id=d534ab3d0a904f0babeb90d2679d1b86)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:14.034288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c307b5e-63d8-4d86-9efa-003cef1cbf1d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:14.050621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39080fe9-6302-4fdd-8411-68a1f1cee600 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:14.054031 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 204/413] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jul 21 06:17:13 2022] PUT /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8/groups/c7746236566f49a58fb70f6b2233fb07/roles/d534ab3d0a904f0babeb90d2679d1b86 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:17:14.057717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98833b6-4d26-4c2e-aa78-324aac483f86 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:14.058237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98833b6-4d26-4c2e-aa78-324aac483f86 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:14.058737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98833b6-4d26-4c2e-aa78-324aac483f86 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:14.060362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 205/414] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:14.067016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e9a80-e055-40b0-b5e0-6a1a64528bee None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:14.067537 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e9a80-e055-40b0-b5e0-6a1a64528bee None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:14.067964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e9a80-e055-40b0-b5e0-6a1a64528bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:14.109800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f59e9a80-e055-40b0-b5e0-6a1a64528bee None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:14.113329 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f59e9a80-e055-40b0-b5e0-6a1a64528bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:14.141641 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e31166-5631-49ee-b456-a9d7760015af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:14.142473 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e31166-5631-49ee-b456-a9d7760015af None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:14.142757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e31166-5631-49ee-b456-a9d7760015af None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:14.143035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e31166-5631-49ee-b456-a9d7760015af None admin] PATH_INFO: `/v3/projects/c098b85eefc14c3cac1c96ebb357b960` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:14.145091 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e31166-5631-49ee-b456-a9d7760015af None admin] RBAC: Authorizing `identity:get_project(project_id=c098b85eefc14c3cac1c96ebb357b960)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:14.146707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e31166-5631-49ee-b456-a9d7760015af None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:14.154092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 210/415] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:13 2022] GET /identity/v3/projects/c098b85eefc14c3cac1c96ebb357b960 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:14.165129 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e31166-5631-49ee-b456-a9d7760015af None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:14.218163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 206/416] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:14.243528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e9a80-e055-40b0-b5e0-6a1a64528bee None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:14.262510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:14.318666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e31166-5631-49ee-b456-a9d7760015af None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:14.329613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:14.375609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e9a80-e055-40b0-b5e0-6a1a64528bee None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:14.416165 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab480a79-c76d-4e53-94cd-b9d51a61e0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:14.417411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b683b0-cc81-4bc2-8096-c7ec337bd9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:14.418250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b683b0-cc81-4bc2-8096-c7ec337bd9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:14.418534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b683b0-cc81-4bc2-8096-c7ec337bd9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:14.418811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b683b0-cc81-4bc2-8096-c7ec337bd9a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:14.420099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab480a79-c76d-4e53-94cd-b9d51a61e0ad None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:14.420384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab480a79-c76d-4e53-94cd-b9d51a61e0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:14.420660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab480a79-c76d-4e53-94cd-b9d51a61e0ad None admin] PATH_INFO: `/v3/projects/c098b85eefc14c3cac1c96ebb357b960/groups/29108e6c5c8c4e09b24a9d3d131f3e75/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:14.423757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab480a79-c76d-4e53-94cd-b9d51a61e0ad None admin] RBAC: Authorizing `identity:create_grant(project_id=c098b85eefc14c3cac1c96ebb357b960, group_id=29108e6c5c8c4e09b24a9d3d131f3e75, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:14.425760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab480a79-c76d-4e53-94cd-b9d51a61e0ad None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:14.439623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b683b0-cc81-4bc2-8096-c7ec337bd9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-19b683b0-cc81-4bc2-8096-c7ec337bd9a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:14.448180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:14.449173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:14.450504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 207/417] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:14.462455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 211/418] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jul 21 06:17:14 2022] PUT /identity/v3/projects/c098b85eefc14c3cac1c96ebb357b960/groups/29108e6c5c8c4e09b24a9d3d131f3e75/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:17:14.463089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b683b0-cc81-4bc2-8096-c7ec337bd9a7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:14.584263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:14.618039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b683b0-cc81-4bc2-8096-c7ec337bd9a7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:14.674295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcefad-9529-435e-ac6d-7316ed83a3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:14.675597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcefad-9529-435e-ac6d-7316ed83a3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:14.675945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcefad-9529-435e-ac6d-7316ed83a3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:14.676231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcefad-9529-435e-ac6d-7316ed83a3d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:14.677007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcefad-9529-435e-ac6d-7316ed83a3d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:14.677316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcefad-9529-435e-ac6d-7316ed83a3d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:14.682674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcefad-9529-435e-ac6d-7316ed83a3d3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:14.694166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 208/419] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:17:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:14.699762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab480a79-c76d-4e53-94cd-b9d51a61e0ad None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:14.816879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab480a79-c76d-4e53-94cd-b9d51a61e0ad None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:14.860650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cbb7e-c820-4148-b767-6d18ae3280b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:14.861663 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cbb7e-c820-4148-b767-6d18ae3280b5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:14.862756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cbb7e-c820-4148-b767-6d18ae3280b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:14.863295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cbb7e-c820-4148-b767-6d18ae3280b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:14.906572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:14.953417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cbb7e-c820-4148-b767-6d18ae3280b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:14.955313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cbb7e-c820-4148-b767-6d18ae3280b5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:15.060769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 212/420] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:15.066838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcefad-9529-435e-ac6d-7316ed83a3d3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:15.190753 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcefad-9529-435e-ac6d-7316ed83a3d3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:15.229353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c958d-cb04-441e-a36a-e1ebc74de149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:15.230239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c958d-cb04-441e-a36a-e1ebc74de149 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:15.230525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c958d-cb04-441e-a36a-e1ebc74de149 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:15.230806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c958d-cb04-441e-a36a-e1ebc74de149 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:15.232002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c958d-cb04-441e-a36a-e1ebc74de149 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:15.233448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c958d-cb04-441e-a36a-e1ebc74de149 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:15.235325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 209/421] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:15 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:15.241560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cbb7e-c820-4148-b767-6d18ae3280b5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:15.257391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91751605-73c8-45a4-98ba-58a3ed0c6cdd None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:15.257686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91751605-73c8-45a4-98ba-58a3ed0c6cdd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:15.257980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91751605-73c8-45a4-98ba-58a3ed0c6cdd None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:15.259606 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 210/422] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:17:15.264514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb61b38-a611-4c7a-94ba-3de7544806ea None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:15.264802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb61b38-a611-4c7a-94ba-3de7544806ea None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:15.265084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb61b38-a611-4c7a-94ba-3de7544806ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:15.274222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:15.304433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5bb61b38-a611-4c7a-94ba-3de7544806ea None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:15.305634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5bb61b38-a611-4c7a-94ba-3de7544806ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:15.351593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:15.426544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cbb7e-c820-4148-b767-6d18ae3280b5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:15.449503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 211/423] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:17:15.480987 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb61b38-a611-4c7a-94ba-3de7544806ea None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:15.482884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7293d7-2901-4c67-8f88-eecafffee65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:15.484264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7293d7-2901-4c67-8f88-eecafffee65f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:15.484749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7293d7-2901-4c67-8f88-eecafffee65f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:15.485217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7293d7-2901-4c67-8f88-eecafffee65f None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:15.486729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7293d7-2901-4c67-8f88-eecafffee65f None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:15.489422 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7293d7-2901-4c67-8f88-eecafffee65f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:15.491577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 213/424] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:15 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:15.498330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7293d7-2901-4c67-8f88-eecafffee65f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:15.671304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7293d7-2901-4c67-8f88-eecafffee65f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:15.683850 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb61b38-a611-4c7a-94ba-3de7544806ea None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:15.746334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:15.748651 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d31341-c4f3-4573-9c58-4873ea3a52f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:15.749322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d31341-c4f3-4573-9c58-4873ea3a52f4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:15.749602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d31341-c4f3-4573-9c58-4873ea3a52f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:15.749880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d31341-c4f3-4573-9c58-4873ea3a52f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:15.750755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d31341-c4f3-4573-9c58-4873ea3a52f4 None admin] RBAC: Adding query filter params (group.id=c7746236566f49a58fb70f6b2233fb07, role.id=d3c75d38368d47d799bf59a0f9c879ea, scope.project.id=2f47c32296eb4caabf57eb7403220ebf) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:15.752923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d31341-c4f3-4573-9c58-4873ea3a52f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:15.753530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d31341-c4f3-4573-9c58-4873ea3a52f4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:15.754309 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63837b-be47-4ef8-a864-e0cd90c6353f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:15.755516 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63837b-be47-4ef8-a864-e0cd90c6353f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:15.755997 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63837b-be47-4ef8-a864-e0cd90c6353f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:15.756288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:15.757087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63837b-be47-4ef8-a864-e0cd90c6353f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:15.767380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 214/425] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 06:17:15 2022] GET /identity/v3/role_assignments?group.id=c7746236566f49a58fb70f6b2233fb07&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 611 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:15.777395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63837b-be47-4ef8-a864-e0cd90c6353f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0e63837b-be47-4ef8-a864-e0cd90c6353f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:15.787760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:15.788744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:15.789524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:15.792329 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 212/426] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:17:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:15.800070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d31341-c4f3-4573-9c58-4873ea3a52f4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:15.897281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a8712-b4a9-43a1-8842-e62f8ecd8ebe None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:15.897602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a8712-b4a9-43a1-8842-e62f8ecd8ebe None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:15.897841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a8712-b4a9-43a1-8842-e62f8ecd8ebe None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:15.899372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 213/427] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:17:15.912704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177eb89-3c7c-453b-b0e7-a8fe99e95675 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:15.913007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177eb89-3c7c-453b-b0e7-a8fe99e95675 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:15.913272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177eb89-3c7c-453b-b0e7-a8fe99e95675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:15.937015 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b177eb89-3c7c-453b-b0e7-a8fe99e95675 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:15.938247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b177eb89-3c7c-453b-b0e7-a8fe99e95675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:15.962760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d31341-c4f3-4573-9c58-4873ea3a52f4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:16.005785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdd556-0ee6-49f6-9c17-e35f009ad1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:16.006421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdd556-0ee6-49f6-9c17-e35f009ad1eb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:16.006707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdd556-0ee6-49f6-9c17-e35f009ad1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:16.006987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdd556-0ee6-49f6-9c17-e35f009ad1eb None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:16.008078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdd556-0ee6-49f6-9c17-e35f009ad1eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:16.008194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdd556-0ee6-49f6-9c17-e35f009ad1eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:16.009717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdd556-0ee6-49f6-9c17-e35f009ad1eb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:16.019342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 215/428] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:16.024609 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdd556-0ee6-49f6-9c17-e35f009ad1eb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:16.044200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 214/429] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:16.071196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177eb89-3c7c-453b-b0e7-a8fe99e95675 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:16.105304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:16.138184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdd556-0ee6-49f6-9c17-e35f009ad1eb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:16.182588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d4107-1429-4205-bbfb-62835b321664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:16.183254 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4107-1429-4205-bbfb-62835b321664 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:16.183544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4107-1429-4205-bbfb-62835b321664 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:16.183828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4107-1429-4205-bbfb-62835b321664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:16.184773 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177eb89-3c7c-453b-b0e7-a8fe99e95675 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:16.209489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:16.223544 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1bfc3-b9d7-4f57-b7ef-a499f7cd06ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:16.223750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1bfc3-b9d7-4f57-b7ef-a499f7cd06ab None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:16.223960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1bfc3-b9d7-4f57-b7ef-a499f7cd06ab None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:16.224454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1bfc3-b9d7-4f57-b7ef-a499f7cd06ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:16.235823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1bfc3-b9d7-4f57-b7ef-a499f7cd06ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-61d1bfc3-b9d7-4f57-b7ef-a499f7cd06ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:16.243630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:16.244718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 06:17:16.245700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:16.251872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 215/430] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:16.253697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1bfc3-b9d7-4f57-b7ef-a499f7cd06ab None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:16.264613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d4107-1429-4205-bbfb-62835b321664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:16.266793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d4107-1429-4205-bbfb-62835b321664 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:16.346975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:16.372795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1bfc3-b9d7-4f57-b7ef-a499f7cd06ab None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:16.402583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 216/431] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 06:17:16.408454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d4107-1429-4205-bbfb-62835b321664 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:16.418303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035df6d-2aa0-4bc8-b56f-9f0ae5bf3bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:16.418981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035df6d-2aa0-4bc8-b56f-9f0ae5bf3bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:16.419265 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035df6d-2aa0-4bc8-b56f-9f0ae5bf3bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:16.419545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035df6d-2aa0-4bc8-b56f-9f0ae5bf3bd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:16.420308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035df6d-2aa0-4bc8-b56f-9f0ae5bf3bd5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:16.420616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035df6d-2aa0-4bc8-b56f-9f0ae5bf3bd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:16.422795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035df6d-2aa0-4bc8-b56f-9f0ae5bf3bd5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:16.440173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 216/432] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:17:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:16.441723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035df6d-2aa0-4bc8-b56f-9f0ae5bf3bd5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:16.496046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:16.553185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d4107-1429-4205-bbfb-62835b321664 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:16.567802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035df6d-2aa0-4bc8-b56f-9f0ae5bf3bd5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:16.602772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ad604-492d-42ed-a1d0-01017f484f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:16.603645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ad604-492d-42ed-a1d0-01017f484f46 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:16.603929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ad604-492d-42ed-a1d0-01017f484f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:16.604215 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ad604-492d-42ed-a1d0-01017f484f46 None admin] PATH_INFO: `/v3/projects/2f47c32296eb4caabf57eb7403220ebf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:16.605845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ad604-492d-42ed-a1d0-01017f484f46 None admin] RBAC: Authorizing `identity:get_project(project_id=2f47c32296eb4caabf57eb7403220ebf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:16.607348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ad604-492d-42ed-a1d0-01017f484f46 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:16.608442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd1922-d15b-49d9-bc0f-3d8a63e726e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:16.609046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 217/433] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:16 2022] GET /identity/v3/projects/2f47c32296eb4caabf57eb7403220ebf => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:16.609438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd1922-d15b-49d9-bc0f-3d8a63e726e1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:16.609868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd1922-d15b-49d9-bc0f-3d8a63e726e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:16.610248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb0c05-9c28-42a1-83be-c3f4a0198e8d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:16.610465 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd1922-d15b-49d9-bc0f-3d8a63e726e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:16.610587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb0c05-9c28-42a1-83be-c3f4a0198e8d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:16.610794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb0c05-9c28-42a1-83be-c3f4a0198e8d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:16.612545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 218/434] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:16.617809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb0c05-9c28-42a1-83be-c3f4a0198e8d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:16.654009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:16.693305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd1922-d15b-49d9-bc0f-3d8a63e726e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:16.694787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd1922-d15b-49d9-bc0f-3d8a63e726e1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:16.731520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb0c05-9c28-42a1-83be-c3f4a0198e8d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:16.743203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:16.769994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ce63a-211a-4e7d-aaaa-86e44ca24308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:16.771414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ce63a-211a-4e7d-aaaa-86e44ca24308 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:16.771698 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ce63a-211a-4e7d-aaaa-86e44ca24308 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:16.771980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ce63a-211a-4e7d-aaaa-86e44ca24308 None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:16.773175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554ce63a-211a-4e7d-aaaa-86e44ca24308 None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:16.774674 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554ce63a-211a-4e7d-aaaa-86e44ca24308 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:16.779161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 219/435] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:16 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:16.780822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72d460-1f47-4992-bda4-63ad7a328a13 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:16.781397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72d460-1f47-4992-bda4-63ad7a328a13 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:16.781514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72d460-1f47-4992-bda4-63ad7a328a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:16.798407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8f72d460-1f47-4992-bda4-63ad7a328a13 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:16.800159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8f72d460-1f47-4992-bda4-63ad7a328a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:16.805292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 217/436] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:16.805917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd1922-d15b-49d9-bc0f-3d8a63e726e1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:16.877554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 220/437] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:16.878431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72d460-1f47-4992-bda4-63ad7a328a13 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:16.909909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:16.922181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd1922-d15b-49d9-bc0f-3d8a63e726e1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:16.960556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15880054-64dc-48cc-90d1-dd3b12139671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:16.961216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15880054-64dc-48cc-90d1-dd3b12139671 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:16.961501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15880054-64dc-48cc-90d1-dd3b12139671 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:16.961780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15880054-64dc-48cc-90d1-dd3b12139671 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:16.962778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15880054-64dc-48cc-90d1-dd3b12139671 None admin] RBAC: Adding query filter params (group.id=c7746236566f49a58fb70f6b2233fb07, role.id=d534ab3d0a904f0babeb90d2679d1b86, scope.project.id=2f47c32296eb4caabf57eb7403220ebf) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:16.963093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15880054-64dc-48cc-90d1-dd3b12139671 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:16.964655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15880054-64dc-48cc-90d1-dd3b12139671 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:16.981968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 218/438] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 06:17:16 2022] GET /identity/v3/role_assignments?group.id=c7746236566f49a58fb70f6b2233fb07&scope.project.id=2f47c32296eb4caabf57eb7403220ebf&role.id=d534ab3d0a904f0babeb90d2679d1b86 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:16.981968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15880054-64dc-48cc-90d1-dd3b12139671 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:16.993383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:16.996192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72d460-1f47-4992-bda4-63ad7a328a13 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:17.051359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51111d58-25ae-448e-8f3e-e0add2f4d39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:17.053260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51111d58-25ae-448e-8f3e-e0add2f4d39f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:17.053712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51111d58-25ae-448e-8f3e-e0add2f4d39f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:17.054550 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51111d58-25ae-448e-8f3e-e0add2f4d39f None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:17.061638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51111d58-25ae-448e-8f3e-e0add2f4d39f None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:17.064152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51111d58-25ae-448e-8f3e-e0add2f4d39f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:17.067065 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 221/439] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:16 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:17.072785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51111d58-25ae-448e-8f3e-e0add2f4d39f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:17.119210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15880054-64dc-48cc-90d1-dd3b12139671 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:17.157539 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78b51f-5f76-434b-bc12-79d19bb8ac8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:17.158920 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78b51f-5f76-434b-bc12-79d19bb8ac8a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:17.159834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78b51f-5f76-434b-bc12-79d19bb8ac8a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:17.160118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78b51f-5f76-434b-bc12-79d19bb8ac8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:17.175267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78b51f-5f76-434b-bc12-79d19bb8ac8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ba78b51f-5f76-434b-bc12-79d19bb8ac8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:17.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:17.184431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:17.185304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 219/440] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:17:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:17.186183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242bdd-57a2-4105-974a-9f48fe9f9651 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:17.186183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242bdd-57a2-4105-974a-9f48fe9f9651 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:17.186536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242bdd-57a2-4105-974a-9f48fe9f9651 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:17.187949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 220/441] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:17.194255 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18242bdd-57a2-4105-974a-9f48fe9f9651 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:17.197687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:17.204492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51111d58-25ae-448e-8f3e-e0add2f4d39f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:17.246109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60abf7-ff47-45c6-abd3-893397eb3ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:17.246980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60abf7-ff47-45c6-abd3-893397eb3ada None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:17.247274 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60abf7-ff47-45c6-abd3-893397eb3ada None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:17.247533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60abf7-ff47-45c6-abd3-893397eb3ada None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:17.248747 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60abf7-ff47-45c6-abd3-893397eb3ada None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:17.250220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60abf7-ff47-45c6-abd3-893397eb3ada None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:17.252223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 222/442] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:17 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:17.253413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819e46e-a826-481c-a9bf-6979c38cc290 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:17.253703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819e46e-a826-481c-a9bf-6979c38cc290 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:17.254064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819e46e-a826-481c-a9bf-6979c38cc290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:17.272022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a819e46e-a826-481c-a9bf-6979c38cc290 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:17.273210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a819e46e-a826-481c-a9bf-6979c38cc290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:17.282444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:17.327535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18242bdd-57a2-4105-974a-9f48fe9f9651 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:17.356166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 223/443] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:17.357114 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819e46e-a826-481c-a9bf-6979c38cc290 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:17.366414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab255463-ef43-479a-a8b3-f3fa0b997120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:17.367417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255463-ef43-479a-a8b3-f3fa0b997120 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:17.367526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255463-ef43-479a-a8b3-f3fa0b997120 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:17.367731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255463-ef43-479a-a8b3-f3fa0b997120 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:17.368568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab255463-ef43-479a-a8b3-f3fa0b997120 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:17.368784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab255463-ef43-479a-a8b3-f3fa0b997120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:17.371413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab255463-ef43-479a-a8b3-f3fa0b997120 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:17.382016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 221/444] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:17.383212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab255463-ef43-479a-a8b3-f3fa0b997120 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:17.454076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:17.461014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819e46e-a826-481c-a9bf-6979c38cc290 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:17.481623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab255463-ef43-479a-a8b3-f3fa0b997120 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:17.495161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd9e71-4458-44e3-8c0f-5dee4be56538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:17.495826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd9e71-4458-44e3-8c0f-5dee4be56538 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:17.496111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd9e71-4458-44e3-8c0f-5dee4be56538 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:17.496390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd9e71-4458-44e3-8c0f-5dee4be56538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:17.497228 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd9e71-4458-44e3-8c0f-5dee4be56538 None admin] RBAC: Adding query filter params (group.id=c7746236566f49a58fb70f6b2233fb07, role.id=d3c75d38368d47d799bf59a0f9c879ea, scope.project.id=860106fdc2dc43498b8d25ed76d618e8) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:17.497642 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd9e71-4458-44e3-8c0f-5dee4be56538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:17.499045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd9e71-4458-44e3-8c0f-5dee4be56538 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:17.505452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 224/445] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 06:17:17 2022] GET /identity/v3/role_assignments?group.id=c7746236566f49a58fb70f6b2233fb07&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=d3c75d38368d47d799bf59a0f9c879ea => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:17.506260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd9e71-4458-44e3-8c0f-5dee4be56538 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:17.515573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:17.525181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43de14-21e4-4198-8e68-85f67274ea0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:17.526646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43de14-21e4-4198-8e68-85f67274ea0a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:17.527085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43de14-21e4-4198-8e68-85f67274ea0a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:17.528073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43de14-21e4-4198-8e68-85f67274ea0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:17.543456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43de14-21e4-4198-8e68-85f67274ea0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1c43de14-21e4-4198-8e68-85f67274ea0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:17.555035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:17.556184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:17.557162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:17.564231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 222/446] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:17.566161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43de14-21e4-4198-8e68-85f67274ea0a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:17.634530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd9e71-4458-44e3-8c0f-5dee4be56538 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:17.679202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d463ec0-4136-4974-9aa5-6023078c2b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:17.679860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d463ec0-4136-4974-9aa5-6023078c2b75 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:17.680142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d463ec0-4136-4974-9aa5-6023078c2b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:17.680420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d463ec0-4136-4974-9aa5-6023078c2b75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:17.687359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43de14-21e4-4198-8e68-85f67274ea0a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:17.712967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:17.725349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ecccd-4e48-4fdf-ac9e-1c1c70dda021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:17.726073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ecccd-4e48-4fdf-ac9e-1c1c70dda021 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:17.726362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ecccd-4e48-4fdf-ac9e-1c1c70dda021 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:17.726828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ecccd-4e48-4fdf-ac9e-1c1c70dda021 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:17.728000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ecccd-4e48-4fdf-ac9e-1c1c70dda021 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:17.728533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ecccd-4e48-4fdf-ac9e-1c1c70dda021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:17.732001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ecccd-4e48-4fdf-ac9e-1c1c70dda021 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:17.743070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 223/447] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:17.748742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ecccd-4e48-4fdf-ac9e-1c1c70dda021 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:17.763997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d463ec0-4136-4974-9aa5-6023078c2b75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:17.765472 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d463ec0-4136-4974-9aa5-6023078c2b75 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:17.814002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:17.871523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ecccd-4e48-4fdf-ac9e-1c1c70dda021 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:17.890389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 225/448] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:17.896430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d463ec0-4136-4974-9aa5-6023078c2b75 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:17.906853 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf5458-991d-41a6-9d96-32b84241678e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:17.907508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf5458-991d-41a6-9d96-32b84241678e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:17.907793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf5458-991d-41a6-9d96-32b84241678e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:17.908288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf5458-991d-41a6-9d96-32b84241678e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:17.954060 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:17.977254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf5458-991d-41a6-9d96-32b84241678e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:17.978748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf5458-991d-41a6-9d96-32b84241678e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:17.995574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d463ec0-4136-4974-9aa5-6023078c2b75 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:18.029150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:18.029530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec7599-99cc-4bd0-9b5a-8d5668a78d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:18.030460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec7599-99cc-4bd0-9b5a-8d5668a78d90 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:18.030929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec7599-99cc-4bd0-9b5a-8d5668a78d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:18.031303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec7599-99cc-4bd0-9b5a-8d5668a78d90 None admin] PATH_INFO: `/v3/projects/860106fdc2dc43498b8d25ed76d618e8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:18.032505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec7599-99cc-4bd0-9b5a-8d5668a78d90 None admin] RBAC: Authorizing `identity:get_project(project_id=860106fdc2dc43498b8d25ed76d618e8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:18.034061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec7599-99cc-4bd0-9b5a-8d5668a78d90 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:18.035749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 226/449] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:17 2022] GET /identity/v3/projects/860106fdc2dc43498b8d25ed76d618e8 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:18.041037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec7599-99cc-4bd0-9b5a-8d5668a78d90 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:18.077542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 224/450] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 06:17:18.083307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf5458-991d-41a6-9d96-32b84241678e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:18.141771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec7599-99cc-4bd0-9b5a-8d5668a78d90 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:18.167071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:18.176492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449856c-5b39-46bc-913c-c9dc0921ea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:18.177355 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449856c-5b39-46bc-913c-c9dc0921ea8f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:18.177639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449856c-5b39-46bc-913c-c9dc0921ea8f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:18.177916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449856c-5b39-46bc-913c-c9dc0921ea8f None admin] PATH_INFO: `/v3/groups/c7746236566f49a58fb70f6b2233fb07` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:18.179141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449856c-5b39-46bc-913c-c9dc0921ea8f None admin] RBAC: Authorizing `identity:get_group(group_id=c7746236566f49a58fb70f6b2233fb07)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:18.180166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf5458-991d-41a6-9d96-32b84241678e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:18.180578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449856c-5b39-46bc-913c-c9dc0921ea8f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:18.182219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 227/451] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:18 2022] GET /identity/v3/groups/c7746236566f49a58fb70f6b2233fb07 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:18.187902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449856c-5b39-46bc-913c-c9dc0921ea8f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:18.215290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f64c5c-2170-49b5-8279-774fbca27bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:18.216154 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f64c5c-2170-49b5-8279-774fbca27bcc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:18.216439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f64c5c-2170-49b5-8279-774fbca27bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:18.216713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f64c5c-2170-49b5-8279-774fbca27bcc None admin] PATH_INFO: `/v3/projects/c098b85eefc14c3cac1c96ebb357b960` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:18.217883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f64c5c-2170-49b5-8279-774fbca27bcc None admin] RBAC: Authorizing `identity:get_project(project_id=c098b85eefc14c3cac1c96ebb357b960)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:18.219340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f64c5c-2170-49b5-8279-774fbca27bcc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:18.221151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 225/452] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:17:18 2022] GET /identity/v3/projects/c098b85eefc14c3cac1c96ebb357b960 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:18.226469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f64c5c-2170-49b5-8279-774fbca27bcc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:18.247760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:18.287849 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449856c-5b39-46bc-913c-c9dc0921ea8f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:18.322585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6391a-54d4-4c5c-8528-1038ef997153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:18.323241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6391a-54d4-4c5c-8528-1038ef997153 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:18.323527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6391a-54d4-4c5c-8528-1038ef997153 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:18.323753 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f64c5c-2170-49b5-8279-774fbca27bcc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:18.323850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6391a-54d4-4c5c-8528-1038ef997153 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:18.324631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6391a-54d4-4c5c-8528-1038ef997153 None admin] RBAC: Adding query filter params (group.id=c7746236566f49a58fb70f6b2233fb07, role.id=d534ab3d0a904f0babeb90d2679d1b86, scope.project.id=860106fdc2dc43498b8d25ed76d618e8) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:18.324955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6391a-54d4-4c5c-8528-1038ef997153 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:18.326451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6391a-54d4-4c5c-8528-1038ef997153 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:18.332383 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 228/453] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 06:17:18 2022] GET /identity/v3/role_assignments?group.id=c7746236566f49a58fb70f6b2233fb07&scope.project.id=860106fdc2dc43498b8d25ed76d618e8&role.id=d534ab3d0a904f0babeb90d2679d1b86 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:18.358536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4cedff-8ffc-4895-a499-81150ec2b6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:18.359420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cedff-8ffc-4895-a499-81150ec2b6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:18.359700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cedff-8ffc-4895-a499-81150ec2b6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:18.359976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cedff-8ffc-4895-a499-81150ec2b6f4 None admin] PATH_INFO: `/v3/groups/29108e6c5c8c4e09b24a9d3d131f3e75` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:18.361796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4cedff-8ffc-4895-a499-81150ec2b6f4 None admin] RBAC: Authorizing `identity:get_group(group_id=29108e6c5c8c4e09b24a9d3d131f3e75)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:18.363299 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4cedff-8ffc-4895-a499-81150ec2b6f4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:18.365002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 226/454] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 06:17:18 2022] GET /identity/v3/groups/29108e6c5c8c4e09b24a9d3d131f3e75 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:18.371155 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6391a-54d4-4c5c-8528-1038ef997153 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:18.433900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:18.477262 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6391a-54d4-4c5c-8528-1038ef997153 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:18.511805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea865cb1-290c-499c-b901-933d7ab2d1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:18.512466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea865cb1-290c-499c-b901-933d7ab2d1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:18.512752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea865cb1-290c-499c-b901-933d7ab2d1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:18.513033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea865cb1-290c-499c-b901-933d7ab2d1c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:18.513866 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea865cb1-290c-499c-b901-933d7ab2d1c9 None admin] RBAC: Adding query filter params (group.id=29108e6c5c8c4e09b24a9d3d131f3e75, role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=c098b85eefc14c3cac1c96ebb357b960) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:18.514224 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea865cb1-290c-499c-b901-933d7ab2d1c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:18.515668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea865cb1-290c-499c-b901-933d7ab2d1c9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:18.521989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 229/455] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 06:17:18 2022] GET /identity/v3/role_assignments?group.id=29108e6c5c8c4e09b24a9d3d131f3e75&scope.project.id=c098b85eefc14c3cac1c96ebb357b960&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:21.089473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20917e2-bff2-45a1-91e4-857591f8c2e3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.090452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20917e2-bff2-45a1-91e4-857591f8c2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.090944 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20917e2-bff2-45a1-91e4-857591f8c2e3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.092787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 227/456] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:21.098867 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afacd5-b5bb-4484-8693-72bb4dc747a2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.099671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afacd5-b5bb-4484-8693-72bb4dc747a2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.100157 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afacd5-b5bb-4484-8693-72bb4dc747a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.117889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-92afacd5-b5bb-4484-8693-72bb4dc747a2 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:21.123454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-92afacd5-b5bb-4484-8693-72bb4dc747a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:21.202432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 230/457] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:21.221261 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec756-191d-4505-b67c-ed53f1e904f9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.221869 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec756-191d-4505-b67c-ed53f1e904f9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.222386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec756-191d-4505-b67c-ed53f1e904f9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.224221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 228/458] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:21.229805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deca099-7267-491a-8d69-f775f7bcfb5a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.230354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deca099-7267-491a-8d69-f775f7bcfb5a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.230836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deca099-7267-491a-8d69-f775f7bcfb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.246161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec756-191d-4505-b67c-ed53f1e904f9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:21.264843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0deca099-7267-491a-8d69-f775f7bcfb5a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:21.266066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0deca099-7267-491a-8d69-f775f7bcfb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:21.348466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:21.369488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:21.379529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 231/459] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:21.385968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988e585-e7be-4dbb-acc6-2fdc09a3fbba None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.385968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988e585-e7be-4dbb-acc6-2fdc09a3fbba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.385968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988e585-e7be-4dbb-acc6-2fdc09a3fbba None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.385968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 232/460] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:21.398209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec756-191d-4505-b67c-ed53f1e904f9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:21.400857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93531-87c1-4a47-994f-f8d84f9624dc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.401141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93531-87c1-4a47-994f-f8d84f9624dc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.401421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93531-87c1-4a47-994f-f8d84f9624dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.423499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-99e93531-87c1-4a47-994f-f8d84f9624dc None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:21.425782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-99e93531-87c1-4a47-994f-f8d84f9624dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:21.447093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e400-c11f-43af-b1a5-1b619698031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:21.447904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e400-c11f-43af-b1a5-1b619698031e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.448188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e400-c11f-43af-b1a5-1b619698031e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.448467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e400-c11f-43af-b1a5-1b619698031e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.460588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102e400-c11f-43af-b1a5-1b619698031e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:21.462365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102e400-c11f-43af-b1a5-1b619698031e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a102e400-c11f-43af-b1a5-1b619698031e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:21.482268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:21.483138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:21.483964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:21.483964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:21.483964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:21.483964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:21.486835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 229/461] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:21.492472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e400-c11f-43af-b1a5-1b619698031e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:21.514035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 233/462] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:17:21.514035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e93531-87c1-4a47-994f-f8d84f9624dc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:21.627665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e400-c11f-43af-b1a5-1b619698031e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:21.639284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e93531-87c1-4a47-994f-f8d84f9624dc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:21.658240 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:21.675854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6d4dc-f1ec-434a-ac0c-9e6073ac2879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:21.676887 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6d4dc-f1ec-434a-ac0c-9e6073ac2879 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.677330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6d4dc-f1ec-434a-ac0c-9e6073ac2879 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.677748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6d4dc-f1ec-434a-ac0c-9e6073ac2879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.687378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:21.690197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbe3fa-8e71-4abb-b346-3352d568c5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:21.690762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6d4dc-f1ec-434a-ac0c-9e6073ac2879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:21.691056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe3fa-8e71-4abb-b346-3352d568c5de None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.691456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe3fa-8e71-4abb-b346-3352d568c5de None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.691972 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe3fa-8e71-4abb-b346-3352d568c5de None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.692942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6d4dc-f1ec-434a-ac0c-9e6073ac2879 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:21.693306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbe3fa-8e71-4abb-b346-3352d568c5de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:21.694346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbe3fa-8e71-4abb-b346-3352d568c5de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:21.696084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbe3fa-8e71-4abb-b346-3352d568c5de None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:21.715431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 234/463] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:21.715431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbe3fa-8e71-4abb-b346-3352d568c5de None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-50e6d4dc-f1ec-434a-ac0c-9e6073ac2879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:21.717782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:21.718747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:21.719646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:21.719646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:21.719646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:21.719646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:21.721823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 230/464] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:21.722817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6d4dc-f1ec-434a-ac0c-9e6073ac2879 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:21.823635 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbe3fa-8e71-4abb-b346-3352d568c5de None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:21.838953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6d4dc-f1ec-434a-ac0c-9e6073ac2879 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:21.859813 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d272b-1c4c-43dc-b671-3df8f81145f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:21.861898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d272b-1c4c-43dc-b671-3df8f81145f2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.862418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d272b-1c4c-43dc-b671-3df8f81145f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.862892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d272b-1c4c-43dc-b671-3df8f81145f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.874315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d272b-1c4c-43dc-b671-3df8f81145f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:21.876096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d272b-1c4c-43dc-b671-3df8f81145f2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:21.877231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ac490-bcc1-4b08-86c1-0357cba08188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:21.877991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ac490-bcc1-4b08-86c1-0357cba08188 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.878230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ac490-bcc1-4b08-86c1-0357cba08188 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.878518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ac490-bcc1-4b08-86c1-0357cba08188 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.879227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ac490-bcc1-4b08-86c1-0357cba08188 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:21.880652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ac490-bcc1-4b08-86c1-0357cba08188 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b54d272b-1c4c-43dc-b671-3df8f81145f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:21.897375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:21.898297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:21.899134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:21.899134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:21.899134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:21.899134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:21.906422 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 235/465] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:21.906609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 231/466] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:17:21 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:17:21.906690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d272b-1c4c-43dc-b671-3df8f81145f2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:21.906758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c3fab-c8dc-4dea-acc4-05ed04efde27 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.906758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c3fab-c8dc-4dea-acc4-05ed04efde27 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.906758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c3fab-c8dc-4dea-acc4-05ed04efde27 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.906758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 232/467] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:21.910429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9eca-3cab-44af-8c33-5f81db23969e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:21.910719 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9eca-3cab-44af-8c33-5f81db23969e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:21.911004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9eca-3cab-44af-8c33-5f81db23969e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:21.930166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3baa9eca-3cab-44af-8c33-5f81db23969e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:21.931629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3baa9eca-3cab-44af-8c33-5f81db23969e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:21.937482 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:21.984855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:22.013731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 233/468] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:22.023011 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa9eca-3cab-44af-8c33-5f81db23969e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:22.057040 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d272b-1c4c-43dc-b671-3df8f81145f2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:22.098444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15030c5-fa37-434a-b74a-9958c5472115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:22.099120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15030c5-fa37-434a-b74a-9958c5472115 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:22.100021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15030c5-fa37-434a-b74a-9958c5472115 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:22.100305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15030c5-fa37-434a-b74a-9958c5472115 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:22.101041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15030c5-fa37-434a-b74a-9958c5472115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:22.101353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15030c5-fa37-434a-b74a-9958c5472115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:22.102776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15030c5-fa37-434a-b74a-9958c5472115 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:22.116169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 236/469] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:22.117015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15030c5-fa37-434a-b74a-9958c5472115 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:22.150843 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa9eca-3cab-44af-8c33-5f81db23969e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:22.194618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88153ea7-67c2-4672-ba44-295042e2ea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:22.195302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88153ea7-67c2-4672-ba44-295042e2ea60 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:22.195844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88153ea7-67c2-4672-ba44-295042e2ea60 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:22.196119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88153ea7-67c2-4672-ba44-295042e2ea60 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:22.197724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88153ea7-67c2-4672-ba44-295042e2ea60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:22.198097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88153ea7-67c2-4672-ba44-295042e2ea60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:22.199482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88153ea7-67c2-4672-ba44-295042e2ea60 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:22.211069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:22.214501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 234/470] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:22.215501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88153ea7-67c2-4672-ba44-295042e2ea60 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:22.240771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15030c5-fa37-434a-b74a-9958c5472115 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:22.270748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:22.278431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1318dd-82ba-40c2-b818-707de6deb9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:22.279573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1318dd-82ba-40c2-b818-707de6deb9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:22.280102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1318dd-82ba-40c2-b818-707de6deb9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:22.280526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1318dd-82ba-40c2-b818-707de6deb9e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:22.292709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1318dd-82ba-40c2-b818-707de6deb9e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:22.294647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1318dd-82ba-40c2-b818-707de6deb9e6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ef1318dd-82ba-40c2-b818-707de6deb9e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:22.315000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:22.315901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:22.316742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:22.316742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:22.316742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:22.316742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:22.319175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 237/471] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:22.320229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1318dd-82ba-40c2-b818-707de6deb9e6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:22.347583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88153ea7-67c2-4672-ba44-295042e2ea60 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:22.385256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb57942-6920-4c8b-98ab-514a00b19921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:22.386381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb57942-6920-4c8b-98ab-514a00b19921 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:22.386891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb57942-6920-4c8b-98ab-514a00b19921 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:22.387373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb57942-6920-4c8b-98ab-514a00b19921 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:22.388282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb57942-6920-4c8b-98ab-514a00b19921 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:22.389926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb57942-6920-4c8b-98ab-514a00b19921 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:22.412125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 235/472] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:17:22 2022] POST /identity/v3/users => generated 248 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:17:22.413165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb57942-6920-4c8b-98ab-514a00b19921 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:22.446619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1318dd-82ba-40c2-b818-707de6deb9e6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:22.494880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca46b2-bbf8-4bd1-99b0-cd75f9e1d317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:22.495861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca46b2-bbf8-4bd1-99b0-cd75f9e1d317 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:22.496426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca46b2-bbf8-4bd1-99b0-cd75f9e1d317 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:22.496850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca46b2-bbf8-4bd1-99b0-cd75f9e1d317 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:22.497888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca46b2-bbf8-4bd1-99b0-cd75f9e1d317 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:22.499568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca46b2-bbf8-4bd1-99b0-cd75f9e1d317 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:22.510018 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:22.529015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 238/473] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:17:22 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:17:22.530494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:22.532019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401658c7-8281-411e-a19e-769daa4fc60a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:22.533137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401658c7-8281-411e-a19e-769daa4fc60a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:22.533740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401658c7-8281-411e-a19e-769daa4fc60a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:22.540065 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 239/474] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:22.541881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d60d9c-3e44-474c-926f-94e745d95d52 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:22.542188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d60d9c-3e44-474c-926f-94e745d95d52 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:22.542471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d60d9c-3e44-474c-926f-94e745d95d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:22.553601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb57942-6920-4c8b-98ab-514a00b19921 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:22.563254 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c1d60d9c-3e44-474c-926f-94e745d95d52 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:22.565676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c1d60d9c-3e44-474c-926f-94e745d95d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:22.616006 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ce195-9720-45c5-8503-22257a5d69b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:22.617058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ce195-9720-45c5-8503-22257a5d69b1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:22.617609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ce195-9720-45c5-8503-22257a5d69b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:22.618127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ce195-9720-45c5-8503-22257a5d69b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:22.619145 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ce195-9720-45c5-8503-22257a5d69b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:22.619658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ce195-9720-45c5-8503-22257a5d69b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:22.621355 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ce195-9720-45c5-8503-22257a5d69b1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:22.640595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 236/475] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:22.643595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ce195-9720-45c5-8503-22257a5d69b1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:22.652976 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 240/476] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:22.678002 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d60d9c-3e44-474c-926f-94e745d95d52 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:22.760913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ce195-9720-45c5-8503-22257a5d69b1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:22.809154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d60d9c-3e44-474c-926f-94e745d95d52 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:22.810302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0856c18-6f22-4ea1-80a1-a44d6226ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:22.811219 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0856c18-6f22-4ea1-80a1-a44d6226ac58 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:22.811708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0856c18-6f22-4ea1-80a1-a44d6226ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:22.812051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0856c18-6f22-4ea1-80a1-a44d6226ac58 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:22.812833 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0856c18-6f22-4ea1-80a1-a44d6226ac58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:22.815624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0856c18-6f22-4ea1-80a1-a44d6226ac58 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:22.843236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:22.845564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 237/477] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:17:22 2022] POST /identity/v3/users => generated 249 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:17:22.865390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324a3f9-99a0-4f80-9957-8ca870acb01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:22.866185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324a3f9-99a0-4f80-9957-8ca870acb01b None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:22.866478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324a3f9-99a0-4f80-9957-8ca870acb01b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:22.866755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324a3f9-99a0-4f80-9957-8ca870acb01b None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:22.867664 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324a3f9-99a0-4f80-9957-8ca870acb01b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:22.869288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324a3f9-99a0-4f80-9957-8ca870acb01b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:22.877070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.sql.core [None req-5324a3f9-99a0-4f80-9957-8ca870acb01b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 06:17:22.877070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 06:17:22.877070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [parameters: {'id': '85948dadcf4c4d7bba8d9ae1c556f037', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 06:17:22.877070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80142) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5324a3f9-99a0-4f80-9957-8ca870acb01b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 21 06:17:22.895688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 06:17:22.896599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application yield resource Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise self.value Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application session.commit() Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 06:17:22.897502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.flush() Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application compat.raise_( Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 06:17:22.899513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 06:17:22.901812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 06:17:22.903783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [parameters: {'id': '85948dadcf4c4d7bba8d9ae1c556f037', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:22.905675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 06:17:22.912061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:22.914054 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 241/478] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jul 21 06:17:22 2022] POST /identity/v3/roles => generated 144 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:22.914192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0856c18-6f22-4ea1-80a1-a44d6226ac58 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:22.918966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:23.040127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0856c18-6f22-4ea1-80a1-a44d6226ac58 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:23.108397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff6cf4-55ad-4741-ae3e-7f70a29a45b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:23.109231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff6cf4-55ad-4741-ae3e-7f70a29a45b6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:23.109518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff6cf4-55ad-4741-ae3e-7f70a29a45b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:23.109801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff6cf4-55ad-4741-ae3e-7f70a29a45b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:23.131911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff6cf4-55ad-4741-ae3e-7f70a29a45b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5dff6cf4-55ad-4741-ae3e-7f70a29a45b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:23.140177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:23.141044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:23.142417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 238/479] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:17:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:23.148610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324a3f9-99a0-4f80-9957-8ca870acb01b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:23.282860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324a3f9-99a0-4f80-9957-8ca870acb01b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:23.321133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a5925-322c-46f3-a484-0a1318058059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:23.321745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a5925-322c-46f3-a484-0a1318058059 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:23.322054 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a5925-322c-46f3-a484-0a1318058059 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:23.322331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a5925-322c-46f3-a484-0a1318058059 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:23.323092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a5925-322c-46f3-a484-0a1318058059 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:23.323396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a5925-322c-46f3-a484-0a1318058059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:23.324790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a5925-322c-46f3-a484-0a1318058059 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:23.335376 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 242/480] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:24.302590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca8cf-8fab-4a0e-9096-f161b01bda67 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:24.303102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca8cf-8fab-4a0e-9096-f161b01bda67 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:24.303188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca8cf-8fab-4a0e-9096-f161b01bda67 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:24.304809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 239/481] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:24.311593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a27e8-ae87-4b50-a1b8-ea73d8becf29 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:24.311987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a27e8-ae87-4b50-a1b8-ea73d8becf29 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:24.312153 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a27e8-ae87-4b50-a1b8-ea73d8becf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:24.333813 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-532a27e8-ae87-4b50-a1b8-ea73d8becf29 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:24.339101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-532a27e8-ae87-4b50-a1b8-ea73d8becf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:24.401643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:24.432265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 243/482] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:24.456967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ca8cf-8fab-4a0e-9096-f161b01bda67 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:24.550252 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:24.596340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ca8cf-8fab-4a0e-9096-f161b01bda67 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:24.640940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68056b7-1dea-4422-b465-79dae04d6646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:24.642898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68056b7-1dea-4422-b465-79dae04d6646 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:24.643180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68056b7-1dea-4422-b465-79dae04d6646 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:24.643455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68056b7-1dea-4422-b465-79dae04d6646 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:24.657485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68056b7-1dea-4422-b465-79dae04d6646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d68056b7-1dea-4422-b465-79dae04d6646 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:24.665446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:24.666480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:24.667288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 240/483] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:24.674199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a27e8-ae87-4b50-a1b8-ea73d8becf29 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:24.805661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a27e8-ae87-4b50-a1b8-ea73d8becf29 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:24.854833 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f76b2e-0833-4f4e-b901-2622b804e49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:24.855427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f76b2e-0833-4f4e-b901-2622b804e49c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:24.855712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f76b2e-0833-4f4e-b901-2622b804e49c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:24.855991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f76b2e-0833-4f4e-b901-2622b804e49c None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:24.856755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f76b2e-0833-4f4e-b901-2622b804e49c None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:24.857064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f76b2e-0833-4f4e-b901-2622b804e49c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:24.858475 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f76b2e-0833-4f4e-b901-2622b804e49c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:24.873246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 244/484] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:24.879397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68056b7-1dea-4422-b465-79dae04d6646 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:25.062709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9aea5-8813-4d01-b18e-c1ea560b6859 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.063653 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9aea5-8813-4d01-b18e-c1ea560b6859 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.063982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9aea5-8813-4d01-b18e-c1ea560b6859 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.065580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 245/485] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:25.070520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65bb30-1537-4d60-b1f2-3851129aaee5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.070806 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65bb30-1537-4d60-b1f2-3851129aaee5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.071086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65bb30-1537-4d60-b1f2-3851129aaee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.097995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8c65bb30-1537-4d60-b1f2-3851129aaee5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:25.099213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8c65bb30-1537-4d60-b1f2-3851129aaee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:25.106997 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68056b7-1dea-4422-b465-79dae04d6646 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:25.156141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:25.163995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:25.171260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6deaf-ae3b-46a0-9194-042eb7b1790f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:25.172071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6deaf-ae3b-46a0-9194-042eb7b1790f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.173072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6deaf-ae3b-46a0-9194-042eb7b1790f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.173529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6deaf-ae3b-46a0-9194-042eb7b1790f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.185085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6deaf-ae3b-46a0-9194-042eb7b1790f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:25.186582 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6deaf-ae3b-46a0-9194-042eb7b1790f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-42c6deaf-ae3b-46a0-9194-042eb7b1790f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:25.210602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:25.211505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:25.212535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:25.212535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:25.212535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:25.212535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:25.212535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 241/486] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:25.222161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6deaf-ae3b-46a0-9194-042eb7b1790f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:25.225108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 246/487] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:17:25.249197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65bb30-1537-4d60-b1f2-3851129aaee5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:25.355613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65bb30-1537-4d60-b1f2-3851129aaee5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:25.394432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdcb3e-4ca2-473e-9d86-1f826d9bc008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:25.395381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdcb3e-4ca2-473e-9d86-1f826d9bc008 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.395487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdcb3e-4ca2-473e-9d86-1f826d9bc008 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.395700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdcb3e-4ca2-473e-9d86-1f826d9bc008 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.396799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6deaf-ae3b-46a0-9194-042eb7b1790f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:25.407210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdcb3e-4ca2-473e-9d86-1f826d9bc008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1bbdcb3e-4ca2-473e-9d86-1f826d9bc008 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:25.415145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:25.416022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:25.417303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 247/488] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:25.422965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd20fb-2351-42ba-b18a-c1282ccef47f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.423328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd20fb-2351-42ba-b18a-c1282ccef47f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.423512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd20fb-2351-42ba-b18a-c1282ccef47f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.425040 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 248/489] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:25.426314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b82c58-a0f4-49e1-b71d-c1001ce5ae4d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.426744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b82c58-a0f4-49e1-b71d-c1001ce5ae4d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.427166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b82c58-a0f4-49e1-b71d-c1001ce5ae4d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.428749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 249/490] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:25.429853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b82c58-a0f4-49e1-b71d-c1001ce5ae4d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:25.436707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a775e9-d407-4974-858b-4fba10912068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:25.437399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a775e9-d407-4974-858b-4fba10912068 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.437689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a775e9-d407-4974-858b-4fba10912068 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.438184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a775e9-d407-4974-858b-4fba10912068 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.438967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a775e9-d407-4974-858b-4fba10912068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:25.439282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a775e9-d407-4974-858b-4fba10912068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:25.440708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a775e9-d407-4974-858b-4fba10912068 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:25.454406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 242/491] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:25.455686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93337286-197d-4bf2-947d-52224ea5342e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.455982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93337286-197d-4bf2-947d-52224ea5342e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.456265 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93337286-197d-4bf2-947d-52224ea5342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.472802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-93337286-197d-4bf2-947d-52224ea5342e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:25.473182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:25.474022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-93337286-197d-4bf2-947d-52224ea5342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:25.511844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:25.557972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 243/492] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:25.557972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5110d-08ba-413b-ab0d-823dea3341a6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.557972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5110d-08ba-413b-ab0d-823dea3341a6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.557972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5110d-08ba-413b-ab0d-823dea3341a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.559082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b82c58-a0f4-49e1-b71d-c1001ce5ae4d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:25.578215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8ab5110d-08ba-413b-ab0d-823dea3341a6 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:25.580107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8ab5110d-08ba-413b-ab0d-823dea3341a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:25.596749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f96da-4d81-4f04-8d10-80b546179d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:25.597811 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f96da-4d81-4f04-8d10-80b546179d7c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.598133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f96da-4d81-4f04-8d10-80b546179d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.598663 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f96da-4d81-4f04-8d10-80b546179d7c None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.599812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f96da-4d81-4f04-8d10-80b546179d7c None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:25.604258 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f96da-4d81-4f04-8d10-80b546179d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:25.605659 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f96da-4d81-4f04-8d10-80b546179d7c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:25.616802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 250/493] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:25.617437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f96da-4d81-4f04-8d10-80b546179d7c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:25.655892 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 244/494] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:25.657612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5110d-08ba-413b-ab0d-823dea3341a6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:25.739166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:25.749745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f96da-4d81-4f04-8d10-80b546179d7c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:25.768611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5110d-08ba-413b-ab0d-823dea3341a6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:25.770130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:25.790427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49001172-9957-4743-ad23-9c511e12fd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:25.791440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49001172-9957-4743-ad23-9c511e12fd42 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.791902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49001172-9957-4743-ad23-9c511e12fd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.792307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49001172-9957-4743-ad23-9c511e12fd42 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.800867 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49001172-9957-4743-ad23-9c511e12fd42 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:25.802353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49001172-9957-4743-ad23-9c511e12fd42 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:25.808170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1bb40-eb26-483f-addb-e030e3f3fb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:25.808973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1bb40-eb26-483f-addb-e030e3f3fb95 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.809285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1bb40-eb26-483f-addb-e030e3f3fb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.809563 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1bb40-eb26-483f-addb-e030e3f3fb95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-49001172-9957-4743-ad23-9c511e12fd42 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:25.818280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:25.819450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:25.820345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:25.820345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:25.820345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:25.820345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:25.820345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:25.820345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:25.820345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:25.820345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:25.820345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:25.820345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:25.820345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 251/495] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 06:17:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:25.821496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49001172-9957-4743-ad23-9c511e12fd42 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:25.825435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1bb40-eb26-483f-addb-e030e3f3fb95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0eb1bb40-eb26-483f-addb-e030e3f3fb95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:25.834033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:25.834929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:25.836059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 245/496] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:25.836911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1bb40-eb26-483f-addb-e030e3f3fb95 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:25.944347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1bb40-eb26-483f-addb-e030e3f3fb95 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:25.944535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49001172-9957-4743-ad23-9c511e12fd42 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:25.979832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d781c6-0d2d-46f5-a75c-6f4789fd2d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:25.980253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0a581-1f0f-4c9d-b6e9-47f398d548f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:25.980637 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d781c6-0d2d-46f5-a75c-6f4789fd2d59 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.980921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d781c6-0d2d-46f5-a75c-6f4789fd2d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.981094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0a581-1f0f-4c9d-b6e9-47f398d548f3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:25.981187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d781c6-0d2d-46f5-a75c-6f4789fd2d59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.981329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0a581-1f0f-4c9d-b6e9-47f398d548f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:25.981587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0a581-1f0f-4c9d-b6e9-47f398d548f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:25.992254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d781c6-0d2d-46f5-a75c-6f4789fd2d59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:25.992742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0a581-1f0f-4c9d-b6e9-47f398d548f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:25.995104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0a581-1f0f-4c9d-b6e9-47f398d548f3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a9d781c6-0d2d-46f5-a75c-6f4789fd2d59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:26.000047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:26.001477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:26.002314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 246/497] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:26.003906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d781c6-0d2d-46f5-a75c-6f4789fd2d59 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-64d0a581-1f0f-4c9d-b6e9-47f398d548f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:26.018888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:26.019761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:26.020617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:26.020617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:26.020617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:26.020617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:26.020617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 252/498] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:26.020886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0a581-1f0f-4c9d-b6e9-47f398d548f3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:26.025603 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:26.036354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:26.125924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d781c6-0d2d-46f5-a75c-6f4789fd2d59 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:26.131917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0a581-1f0f-4c9d-b6e9-47f398d548f3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:26.162467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26030138-69ff-4f55-a5c0-40c299da6749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:26.163366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26030138-69ff-4f55-a5c0-40c299da6749 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.163797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26030138-69ff-4f55-a5c0-40c299da6749 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.164247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26030138-69ff-4f55-a5c0-40c299da6749 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:26.165229 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26030138-69ff-4f55-a5c0-40c299da6749 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:26.165683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26030138-69ff-4f55-a5c0-40c299da6749 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:26.167290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26030138-69ff-4f55-a5c0-40c299da6749 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:26.168273 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293f58e-150e-4368-b118-c3b745786800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:26.168934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293f58e-150e-4368-b118-c3b745786800 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.169214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293f58e-150e-4368-b118-c3b745786800 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.169488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293f58e-150e-4368-b118-c3b745786800 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:26.170265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293f58e-150e-4368-b118-c3b745786800 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:26.170571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293f58e-150e-4368-b118-c3b745786800 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:26.171923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293f58e-150e-4368-b118-c3b745786800 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:26.181982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 253/499] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:26.181982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293f58e-150e-4368-b118-c3b745786800 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:26.216735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 247/500] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 06:17:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:26.216735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26030138-69ff-4f55-a5c0-40c299da6749 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:26.255294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:26.291267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293f58e-150e-4368-b118-c3b745786800 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:26.302924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:26.324749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26030138-69ff-4f55-a5c0-40c299da6749 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:26.328487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8f0ed-0fb9-4a58-9d80-459aa0cbec63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:26.329162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8f0ed-0fb9-4a58-9d80-459aa0cbec63 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.329446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8f0ed-0fb9-4a58-9d80-459aa0cbec63 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.329724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8f0ed-0fb9-4a58-9d80-459aa0cbec63 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:26.330507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8f0ed-0fb9-4a58-9d80-459aa0cbec63 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:26.330821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8f0ed-0fb9-4a58-9d80-459aa0cbec63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:26.332203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8f0ed-0fb9-4a58-9d80-459aa0cbec63 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:26.340463 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 254/501] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:26.341263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8f0ed-0fb9-4a58-9d80-459aa0cbec63 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:26.366733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198dc2a-b2d3-4e03-a459-3d196c4f3514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:26.367409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198dc2a-b2d3-4e03-a459-3d196c4f3514 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.367698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198dc2a-b2d3-4e03-a459-3d196c4f3514 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.367981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198dc2a-b2d3-4e03-a459-3d196c4f3514 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:26.368717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198dc2a-b2d3-4e03-a459-3d196c4f3514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:26.369161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198dc2a-b2d3-4e03-a459-3d196c4f3514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:26.370450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198dc2a-b2d3-4e03-a459-3d196c4f3514 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:26.382646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 248/502] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:26.386339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198dc2a-b2d3-4e03-a459-3d196c4f3514 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:26.447861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8f0ed-0fb9-4a58-9d80-459aa0cbec63 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:26.473522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:26.481461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198dc2a-b2d3-4e03-a459-3d196c4f3514 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:26.482374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f364b-4bb4-4699-8759-217cfff00905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:26.483167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f364b-4bb4-4699-8759-217cfff00905 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.483447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f364b-4bb4-4699-8759-217cfff00905 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.483729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f364b-4bb4-4699-8759-217cfff00905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:26.495063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f364b-4bb4-4699-8759-217cfff00905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:26.496519 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f364b-4bb4-4699-8759-217cfff00905 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-cf4f364b-4bb4-4699-8759-217cfff00905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:26.514011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:26.514967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:26.515912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:26.515912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:26.515912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:26.515912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:26.516131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3eee6-a5b9-4f00-bff5-9ed0a4b98e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:26.516217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 255/503] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:26.516698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3eee6-a5b9-4f00-bff5-9ed0a4b98e88 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.516698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3eee6-a5b9-4f00-bff5-9ed0a4b98e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.516698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3eee6-a5b9-4f00-bff5-9ed0a4b98e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:26.516878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f364b-4bb4-4699-8759-217cfff00905 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:26.526701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:26.583958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3eee6-a5b9-4f00-bff5-9ed0a4b98e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:26.585429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3eee6-a5b9-4f00-bff5-9ed0a4b98e88 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:26.618597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f364b-4bb4-4699-8759-217cfff00905 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:26.652349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ef374-2372-49b3-8247-941c4f67277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:26.653141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef374-2372-49b3-8247-941c4f67277c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.653431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef374-2372-49b3-8247-941c4f67277c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.653707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef374-2372-49b3-8247-941c4f67277c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:26.664111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ef374-2372-49b3-8247-941c4f67277c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:26.665562 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ef374-2372-49b3-8247-941c4f67277c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fd0ef374-2372-49b3-8247-941c4f67277c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:26.682566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:26.683430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:26.684287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:26.684287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:26.684287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:26.684287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:26.684509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 249/504] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:26.684509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9244b-bdec-4267-9184-f4f328cbccc3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.684509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9244b-bdec-4267-9184-f4f328cbccc3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.684959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9244b-bdec-4267-9184-f4f328cbccc3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:26.685198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 256/505] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:26.689989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 250/506] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:26.689989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9244b-bdec-4267-9184-f4f328cbccc3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:26.690393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ef374-2372-49b3-8247-941c4f67277c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:26.721094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:26.732713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:26.794381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ef374-2372-49b3-8247-941c4f67277c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:26.798514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9244b-bdec-4267-9184-f4f328cbccc3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:26.827527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc532b-3ac9-45f8-a2ac-2a9688155d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:26.828966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc532b-3ac9-45f8-a2ac-2a9688155d32 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.829256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc532b-3ac9-45f8-a2ac-2a9688155d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.829533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc532b-3ac9-45f8-a2ac-2a9688155d32 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:26.832554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b109b-32d0-4e90-a3a0-97d9093f79d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:26.833207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b109b-32d0-4e90-a3a0-97d9093f79d0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.833488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b109b-32d0-4e90-a3a0-97d9093f79d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.833760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b109b-32d0-4e90-a3a0-97d9093f79d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:26.834514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b109b-32d0-4e90-a3a0-97d9093f79d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:26.834820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b109b-32d0-4e90-a3a0-97d9093f79d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:26.836186 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b109b-32d0-4e90-a3a0-97d9093f79d0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:26.837069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc532b-3ac9-45f8-a2ac-2a9688155d32 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:26.838540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc532b-3ac9-45f8-a2ac-2a9688155d32 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:26.847559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 251/507] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:26.848362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b109b-32d0-4e90-a3a0-97d9093f79d0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c6bc532b-3ac9-45f8-a2ac-2a9688155d32 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:26.852779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:26.853695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:26.854592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:26.854592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:26.854592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:26.854592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:26.854592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:26.854592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:26.854592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:26.854592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:26.854592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 06:17:26.854592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:26.854592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 257/508] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:26.855142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc532b-3ac9-45f8-a2ac-2a9688155d32 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:26.947069 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:26.949016 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:26.950232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b109b-32d0-4e90-a3a0-97d9093f79d0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:26.959989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc532b-3ac9-45f8-a2ac-2a9688155d32 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:26.986277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7a4cc-14d4-4ae7-880f-9964527dcdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:26.986873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7a4cc-14d4-4ae7-880f-9964527dcdae None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.987150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7a4cc-14d4-4ae7-880f-9964527dcdae None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.987425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7a4cc-14d4-4ae7-880f-9964527dcdae None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:26.988141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7a4cc-14d4-4ae7-880f-9964527dcdae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:26.988447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7a4cc-14d4-4ae7-880f-9964527dcdae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:26.989810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7a4cc-14d4-4ae7-880f-9964527dcdae None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:26.993234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070baf0-32ed-4ff6-bbba-1fe6e3fb5abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:26.994052 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070baf0-32ed-4ff6-bbba-1fe6e3fb5abf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:26.994341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070baf0-32ed-4ff6-bbba-1fe6e3fb5abf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:26.994622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070baf0-32ed-4ff6-bbba-1fe6e3fb5abf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.001995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 252/509] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:27.003175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976795c-fe74-498e-a663-12ba8bb62bd4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:27.003458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976795c-fe74-498e-a663-12ba8bb62bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:27.003731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976795c-fe74-498e-a663-12ba8bb62bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.008630 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070baf0-32ed-4ff6-bbba-1fe6e3fb5abf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:27.010123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070baf0-32ed-4ff6-bbba-1fe6e3fb5abf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:27.020305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b976795c-fe74-498e-a663-12ba8bb62bd4 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:27.021474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b976795c-fe74-498e-a663-12ba8bb62bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9070baf0-32ed-4ff6-bbba-1fe6e3fb5abf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:27.027852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:27.028805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:27.029666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:27.029666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:27.029666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:27.029666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:27.029666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 258/510] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:27.029926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070baf0-32ed-4ff6-bbba-1fe6e3fb5abf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:27.097098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 253/511] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:27.097962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976795c-fe74-498e-a663-12ba8bb62bd4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:27.146546 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070baf0-32ed-4ff6-bbba-1fe6e3fb5abf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:27.180759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e522c0-af6f-404c-9c0f-4ab598831ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:27.181647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e522c0-af6f-404c-9c0f-4ab598831ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:27.181955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e522c0-af6f-404c-9c0f-4ab598831ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:27.182243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e522c0-af6f-404c-9c0f-4ab598831ff2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.189580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:27.189720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e522c0-af6f-404c-9c0f-4ab598831ff2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:27.191205 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e522c0-af6f-404c-9c0f-4ab598831ff2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:27.199686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976795c-fe74-498e-a663-12ba8bb62bd4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d1e522c0-af6f-404c-9c0f-4ab598831ff2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:27.205771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:27.206700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:27.207611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:27.207611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:27.207611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:27.207611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:27.207611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:27.207611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:27.207611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:27.207611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:27.207611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 06:17:27.207611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:27.207611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 259/512] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:27.208726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e522c0-af6f-404c-9c0f-4ab598831ff2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:27.211143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:27.239951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ad997-e8b8-4da0-ad7e-9bbb2cb76ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:27.240576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad997-e8b8-4da0-ad7e-9bbb2cb76ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:27.240862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad997-e8b8-4da0-ad7e-9bbb2cb76ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:27.241137 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad997-e8b8-4da0-ad7e-9bbb2cb76ef7 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.241992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ad997-e8b8-4da0-ad7e-9bbb2cb76ef7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:27.242304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ad997-e8b8-4da0-ad7e-9bbb2cb76ef7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:27.243743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ad997-e8b8-4da0-ad7e-9bbb2cb76ef7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:27.260838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 254/513] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 06:17:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:27.261629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ad997-e8b8-4da0-ad7e-9bbb2cb76ef7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:27.310835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e522c0-af6f-404c-9c0f-4ab598831ff2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:27.346062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bb774-bf55-4084-bd4e-dee70e8cfdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:27.346859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bb774-bf55-4084-bd4e-dee70e8cfdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:27.347141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bb774-bf55-4084-bd4e-dee70e8cfdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:27.347416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bb774-bf55-4084-bd4e-dee70e8cfdc2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.354982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bb774-bf55-4084-bd4e-dee70e8cfdc2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:27.356432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bb774-bf55-4084-bd4e-dee70e8cfdc2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:27.364242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ad997-e8b8-4da0-ad7e-9bbb2cb76ef7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-529bb774-bf55-4084-bd4e-dee70e8cfdc2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:27.371141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:27.372218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 260/514] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:27.373387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bb774-bf55-4084-bd4e-dee70e8cfdc2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:27.400553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f175a7-9461-4389-a4be-138e8733556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:27.401623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f175a7-9461-4389-a4be-138e8733556c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:27.401914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f175a7-9461-4389-a4be-138e8733556c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:27.402221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f175a7-9461-4389-a4be-138e8733556c None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.402947 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f175a7-9461-4389-a4be-138e8733556c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:27.403258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f175a7-9461-4389-a4be-138e8733556c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:27.404640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f175a7-9461-4389-a4be-138e8733556c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:27.416217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 255/515] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:27.417000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f175a7-9461-4389-a4be-138e8733556c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:27.428744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:27.437960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:27.476491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bb774-bf55-4084-bd4e-dee70e8cfdc2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:27.509601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a719b-55c4-4f1b-bfa4-a1060f1724b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:27.510352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a719b-55c4-4f1b-bfa4-a1060f1724b3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:27.510635 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a719b-55c4-4f1b-bfa4-a1060f1724b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:27.510910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a719b-55c4-4f1b-bfa4-a1060f1724b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.517203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f175a7-9461-4389-a4be-138e8733556c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:27.521869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a719b-55c4-4f1b-bfa4-a1060f1724b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:27.523361 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a719b-55c4-4f1b-bfa4-a1060f1724b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-047a719b-55c4-4f1b-bfa4-a1060f1724b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:27.541669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:27.542634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:27.543473 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:27.543473 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:27.543473 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:27.543473 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:27.543473 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 261/516] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:27.543724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a719b-55c4-4f1b-bfa4-a1060f1724b3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:27.556025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512548-adf7-46cf-babd-5ac0ec4f265d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:27.556695 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512548-adf7-46cf-babd-5ac0ec4f265d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:27.556982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512548-adf7-46cf-babd-5ac0ec4f265d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:27.557263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512548-adf7-46cf-babd-5ac0ec4f265d None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.558147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512548-adf7-46cf-babd-5ac0ec4f265d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:27.558462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512548-adf7-46cf-babd-5ac0ec4f265d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:27.559902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512548-adf7-46cf-babd-5ac0ec4f265d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:27.577659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 256/517] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Jul 21 06:17:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:27.578592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512548-adf7-46cf-babd-5ac0ec4f265d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:27.641382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:27.646799 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a719b-55c4-4f1b-bfa4-a1060f1724b3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:27.665661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:27.676755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512548-adf7-46cf-babd-5ac0ec4f265d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:27.684134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597df3a6-6bd9-4c50-aba4-9ed04465e138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:27.684780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df3a6-6bd9-4c50-aba4-9ed04465e138 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:27.685064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df3a6-6bd9-4c50-aba4-9ed04465e138 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:27.685342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df3a6-6bd9-4c50-aba4-9ed04465e138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.710220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e459905-bb56-4f3f-b1fc-13c1b22ec116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:27.710886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e459905-bb56-4f3f-b1fc-13c1b22ec116 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:27.711195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e459905-bb56-4f3f-b1fc-13c1b22ec116 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:27.711474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e459905-bb56-4f3f-b1fc-13c1b22ec116 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.712307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e459905-bb56-4f3f-b1fc-13c1b22ec116 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:27.712615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e459905-bb56-4f3f-b1fc-13c1b22ec116 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:27.714061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e459905-bb56-4f3f-b1fc-13c1b22ec116 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:27.731300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 257/518] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 06:17:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:27.732090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e459905-bb56-4f3f-b1fc-13c1b22ec116 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:27.755113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597df3a6-6bd9-4c50-aba4-9ed04465e138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:27.756586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597df3a6-6bd9-4c50-aba4-9ed04465e138 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:27.847437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e459905-bb56-4f3f-b1fc-13c1b22ec116 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:27.855975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 262/519] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:27.856811 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597df3a6-6bd9-4c50-aba4-9ed04465e138 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:27.873336 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:27.876751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:27.887752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bbb70-28ef-49ec-8519-6be3fa4faeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:27.888541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bbb70-28ef-49ec-8519-6be3fa4faeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:27.888824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bbb70-28ef-49ec-8519-6be3fa4faeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:27.889100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bbb70-28ef-49ec-8519-6be3fa4faeb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.896683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bbb70-28ef-49ec-8519-6be3fa4faeb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:27.898162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bbb70-28ef-49ec-8519-6be3fa4faeb8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a32bbb70-28ef-49ec-8519-6be3fa4faeb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:27.913352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:27.914240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:27.915092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:27.915092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:27.915092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:27.915092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:27.915092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:27.915092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:27.915092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:27.915092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:27.915092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 258/520] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:27.915573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bbb70-28ef-49ec-8519-6be3fa4faeb8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:27.957079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597df3a6-6bd9-4c50-aba4-9ed04465e138 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:27.992187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f5294-6112-4f0f-9b8e-bf53438fbf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:27.992857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f5294-6112-4f0f-9b8e-bf53438fbf37 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:27.993173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f5294-6112-4f0f-9b8e-bf53438fbf37 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:27.993449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f5294-6112-4f0f-9b8e-bf53438fbf37 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:27.994203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f5294-6112-4f0f-9b8e-bf53438fbf37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:27.994517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f5294-6112-4f0f-9b8e-bf53438fbf37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:27.995913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f5294-6112-4f0f-9b8e-bf53438fbf37 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:28.007752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 263/521] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:28.008740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f5294-6112-4f0f-9b8e-bf53438fbf37 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:28.014531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bbb70-28ef-49ec-8519-6be3fa4faeb8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:28.049295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f52150-0da2-4f6b-9ce9-395cd5452a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:28.049960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f52150-0da2-4f6b-9ce9-395cd5452a8a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:28.050262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f52150-0da2-4f6b-9ce9-395cd5452a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:28.050547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f52150-0da2-4f6b-9ce9-395cd5452a8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:28.091122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:28.102504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:28.115010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f5294-6112-4f0f-9b8e-bf53438fbf37 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:28.118654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f52150-0da2-4f6b-9ce9-395cd5452a8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:28.120123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f52150-0da2-4f6b-9ce9-395cd5452a8a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:28.149628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621abec-f09d-4178-a55c-9194f3037dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:28.150305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621abec-f09d-4178-a55c-9194f3037dba None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:28.150584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621abec-f09d-4178-a55c-9194f3037dba None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:28.151116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621abec-f09d-4178-a55c-9194f3037dba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:28.218128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 259/522] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:28.218290 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621abec-f09d-4178-a55c-9194f3037dba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:28.218991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f52150-0da2-4f6b-9ce9-395cd5452a8a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:28.220090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621abec-f09d-4178-a55c-9194f3037dba None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:28.295729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:28.302546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:28.319053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f52150-0da2-4f6b-9ce9-395cd5452a8a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:28.322383 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 264/523] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:28.323198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621abec-f09d-4178-a55c-9194f3037dba None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:28.353449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c56b4-ddd0-4563-a218-4822510a54d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:28.354197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c56b4-ddd0-4563-a218-4822510a54d2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:28.354488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c56b4-ddd0-4563-a218-4822510a54d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:28.354768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c56b4-ddd0-4563-a218-4822510a54d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:28.364688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c56b4-ddd0-4563-a218-4822510a54d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:28.366182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c56b4-ddd0-4563-a218-4822510a54d2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ec0c56b4-ddd0-4563-a218-4822510a54d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:28.384036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:28.384980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:28.385972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:28.385972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:28.385972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:28.385972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:28.385972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 260/524] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:28.385972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c56b4-ddd0-4563-a218-4822510a54d2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:28.427201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621abec-f09d-4178-a55c-9194f3037dba None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:28.460767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ec498-cb68-43ca-89d4-50b872bea740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:28.461419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ec498-cb68-43ca-89d4-50b872bea740 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:28.461705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ec498-cb68-43ca-89d4-50b872bea740 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:28.462003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ec498-cb68-43ca-89d4-50b872bea740 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:28.462795 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ec498-cb68-43ca-89d4-50b872bea740 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:28.463104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ec498-cb68-43ca-89d4-50b872bea740 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:28.464536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ec498-cb68-43ca-89d4-50b872bea740 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:28.481736 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 265/525] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:28.482564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ec498-cb68-43ca-89d4-50b872bea740 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:28.491373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c56b4-ddd0-4563-a218-4822510a54d2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:28.518290 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:28.525207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93093c07-525f-4650-b368-c9a2a5cab104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:28.526177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093c07-525f-4650-b368-c9a2a5cab104 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:28.526464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093c07-525f-4650-b368-c9a2a5cab104 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:28.526741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093c07-525f-4650-b368-c9a2a5cab104 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:28.527435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93093c07-525f-4650-b368-c9a2a5cab104 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:28.528901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93093c07-525f-4650-b368-c9a2a5cab104 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:28.536986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:28.550221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 261/526] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:17:28 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:17:28.551011 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93093c07-525f-4650-b368-c9a2a5cab104 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:28.591999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ec498-cb68-43ca-89d4-50b872bea740 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:28.631746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef0891-4f48-4c2e-8677-c2d6bf7ed431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:28.632470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef0891-4f48-4c2e-8677-c2d6bf7ed431 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:28.632756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef0891-4f48-4c2e-8677-c2d6bf7ed431 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:28.633033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef0891-4f48-4c2e-8677-c2d6bf7ed431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:28.644944 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef0891-4f48-4c2e-8677-c2d6bf7ed431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:28.646489 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef0891-4f48-4c2e-8677-c2d6bf7ed431 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f6ef0891-4f48-4c2e-8677-c2d6bf7ed431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:28.666850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:28.667797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:28.668687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:28.668687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:28.668687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:28.668687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:28.668687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 266/527] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:28.668980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef0891-4f48-4c2e-8677-c2d6bf7ed431 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:28.682356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93093c07-525f-4650-b368-c9a2a5cab104 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:28.721961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b098d88-9179-49cc-94dc-40fc1120ee0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:28.722788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b098d88-9179-49cc-94dc-40fc1120ee0c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:28.723075 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b098d88-9179-49cc-94dc-40fc1120ee0c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:28.723356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b098d88-9179-49cc-94dc-40fc1120ee0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:28.735737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b098d88-9179-49cc-94dc-40fc1120ee0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:28.737198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b098d88-9179-49cc-94dc-40fc1120ee0c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2b098d88-9179-49cc-94dc-40fc1120ee0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:28.756408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:28.757373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:28.758312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:28.758312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:28.758312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:28.758312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:28.758312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 262/528] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:28.758611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b098d88-9179-49cc-94dc-40fc1120ee0c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:28.783587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:28.799724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef0891-4f48-4c2e-8677-c2d6bf7ed431 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:28.810641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:28.840138 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660338fb-0c33-4e54-bb19-afe418f969ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:28.840797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660338fb-0c33-4e54-bb19-afe418f969ac None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:28.841139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660338fb-0c33-4e54-bb19-afe418f969ac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:28.841419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660338fb-0c33-4e54-bb19-afe418f969ac None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:28.842183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660338fb-0c33-4e54-bb19-afe418f969ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:28.842494 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660338fb-0c33-4e54-bb19-afe418f969ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:28.844677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660338fb-0c33-4e54-bb19-afe418f969ac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:28.856318 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 267/529] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:28.856948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660338fb-0c33-4e54-bb19-afe418f969ac None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:28.880389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b098d88-9179-49cc-94dc-40fc1120ee0c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:28.916367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0b89f-0fcb-4c1c-b35f-fd4cc0b18e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:28.917247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0b89f-0fcb-4c1c-b35f-fd4cc0b18e43 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:28.917585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0b89f-0fcb-4c1c-b35f-fd4cc0b18e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:28.917806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0b89f-0fcb-4c1c-b35f-fd4cc0b18e43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:28.918674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0b89f-0fcb-4c1c-b35f-fd4cc0b18e43 None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=058f099e6f7545a1b2734dbe5afd2027) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:28.918985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0b89f-0fcb-4c1c-b35f-fd4cc0b18e43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:28.920521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0b89f-0fcb-4c1c-b35f-fd4cc0b18e43 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:28.926959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 263/530] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:28 2022] GET /identity/v3/role_assignments?user.id=058f099e6f7545a1b2734dbe5afd2027&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:28.927762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0b89f-0fcb-4c1c-b35f-fd4cc0b18e43 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:28.998632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660338fb-0c33-4e54-bb19-afe418f969ac None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:29.040167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda4e98-3829-4f74-ae2f-1b7e48a06308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:29.040845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda4e98-3829-4f74-ae2f-1b7e48a06308 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:29.041135 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda4e98-3829-4f74-ae2f-1b7e48a06308 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:29.041415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda4e98-3829-4f74-ae2f-1b7e48a06308 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:29.042207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda4e98-3829-4f74-ae2f-1b7e48a06308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:29.042525 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda4e98-3829-4f74-ae2f-1b7e48a06308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:29.043940 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda4e98-3829-4f74-ae2f-1b7e48a06308 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:29.070964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0b89f-0fcb-4c1c-b35f-fd4cc0b18e43 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:29.077564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 268/531] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:29.077564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda4e98-3829-4f74-ae2f-1b7e48a06308 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:29.088391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:29.094740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:29.123854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f64c95-bc6f-4c02-a1fb-4cc8f67414d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:29.125397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f64c95-bc6f-4c02-a1fb-4cc8f67414d9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:29.126030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f64c95-bc6f-4c02-a1fb-4cc8f67414d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:29.126416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f64c95-bc6f-4c02-a1fb-4cc8f67414d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:29.127275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f64c95-bc6f-4c02-a1fb-4cc8f67414d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:29.127587 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f64c95-bc6f-4c02-a1fb-4cc8f67414d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:29.128969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f64c95-bc6f-4c02-a1fb-4cc8f67414d9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:29.144397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 264/532] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:29.145258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f64c95-bc6f-4c02-a1fb-4cc8f67414d9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:29.222892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda4e98-3829-4f74-ae2f-1b7e48a06308 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:29.265831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f64c95-bc6f-4c02-a1fb-4cc8f67414d9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:29.270638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ab277-9054-4dfa-8572-306cc8df17df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:29.271621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ab277-9054-4dfa-8572-306cc8df17df None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:29.272070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ab277-9054-4dfa-8572-306cc8df17df None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:29.272487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ab277-9054-4dfa-8572-306cc8df17df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:29.281997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ab277-9054-4dfa-8572-306cc8df17df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:29.284390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ab277-9054-4dfa-8572-306cc8df17df None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-da4ab277-9054-4dfa-8572-306cc8df17df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:29.302218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:29.303303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:29.304276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:29.304276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:29.304276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:29.304276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:29.304276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:29.304276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:29.304276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:29.304276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:29.304276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 269/533] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:29.304836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ab277-9054-4dfa-8572-306cc8df17df None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:29.316988 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55063c-ae52-440e-93f9-89b47581b0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:29.318146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55063c-ae52-440e-93f9-89b47581b0fe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:29.318599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55063c-ae52-440e-93f9-89b47581b0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:29.319033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55063c-ae52-440e-93f9-89b47581b0fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:29.326608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55063c-ae52-440e-93f9-89b47581b0fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:29.329220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55063c-ae52-440e-93f9-89b47581b0fe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2e55063c-ae52-440e-93f9-89b47581b0fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:29.347514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:29.348406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:29.349230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:29.349230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:29.349230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:29.349230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:29.349230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:29.349230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:29.349230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:29.349230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:29.351026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 265/534] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:29.352081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55063c-ae52-440e-93f9-89b47581b0fe None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:29.382373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:29.384028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:29.425565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ab277-9054-4dfa-8572-306cc8df17df None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:29.461191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55063c-ae52-440e-93f9-89b47581b0fe None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:29.464818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c7233-6894-4ef9-9082-6cb0c3aff2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:29.465695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c7233-6894-4ef9-9082-6cb0c3aff2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:29.466088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c7233-6894-4ef9-9082-6cb0c3aff2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:29.466443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c7233-6894-4ef9-9082-6cb0c3aff2c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:29.474289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c7233-6894-4ef9-9082-6cb0c3aff2c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:29.475896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c7233-6894-4ef9-9082-6cb0c3aff2c8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ac3c7233-6894-4ef9-9082-6cb0c3aff2c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:29.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:29.493438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:29.494524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:29.494524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:29.494524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:29.494524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:29.494524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:29.494524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:29.494524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:29.494524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:29.495509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 270/535] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:29.496279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c7233-6894-4ef9-9082-6cb0c3aff2c8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:29.502368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4bdc4-0b5f-4580-b860-e732611403fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:29.503272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4bdc4-0b5f-4580-b860-e732611403fe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:29.503719 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4bdc4-0b5f-4580-b860-e732611403fe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:29.504195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4bdc4-0b5f-4580-b860-e732611403fe None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:29.505149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4bdc4-0b5f-4580-b860-e732611403fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:29.505666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4bdc4-0b5f-4580-b860-e732611403fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:29.507336 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4bdc4-0b5f-4580-b860-e732611403fe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:29.531507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 266/536] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:29.533039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4bdc4-0b5f-4580-b860-e732611403fe None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:29.605408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c7233-6894-4ef9-9082-6cb0c3aff2c8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:29.623957 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:29.636653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4bdc4-0b5f-4580-b860-e732611403fe None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:29.638300 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:29.646589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fe9c9-e920-4401-b4d1-3c399eabdaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:29.647494 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fe9c9-e920-4401-b4d1-3c399eabdaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:29.647912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fe9c9-e920-4401-b4d1-3c399eabdaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:29.648297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fe9c9-e920-4401-b4d1-3c399eabdaa1 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:29.649467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fe9c9-e920-4401-b4d1-3c399eabdaa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:29.649885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fe9c9-e920-4401-b4d1-3c399eabdaa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:29.651737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fe9c9-e920-4401-b4d1-3c399eabdaa1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:29.665591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 271/537] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:29.666673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fe9c9-e920-4401-b4d1-3c399eabdaa1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:29.675472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ea30e-4e74-4991-816b-d9bea4abcc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:29.676345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ea30e-4e74-4991-816b-d9bea4abcc73 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:29.676777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ea30e-4e74-4991-816b-d9bea4abcc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:29.677223 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ea30e-4e74-4991-816b-d9bea4abcc73 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:29.678190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ea30e-4e74-4991-816b-d9bea4abcc73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:29.678643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ea30e-4e74-4991-816b-d9bea4abcc73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:29.680221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ea30e-4e74-4991-816b-d9bea4abcc73 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:29.693547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 267/538] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:29.694737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ea30e-4e74-4991-816b-d9bea4abcc73 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:29.780383 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fe9c9-e920-4401-b4d1-3c399eabdaa1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:29.802154 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ea30e-4e74-4991-816b-d9bea4abcc73 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:29.830633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18bbc5-e6c8-40f5-ad48-54d0511c1f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:29.831397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18bbc5-e6c8-40f5-ad48-54d0511c1f5c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:29.831689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18bbc5-e6c8-40f5-ad48-54d0511c1f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:29.831969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18bbc5-e6c8-40f5-ad48-54d0511c1f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:29.832921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18bbc5-e6c8-40f5-ad48-54d0511c1f5c None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=fa2c8be3105a415ead65579dc18ba40d) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:29.833241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18bbc5-e6c8-40f5-ad48-54d0511c1f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:29.834700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18bbc5-e6c8-40f5-ad48-54d0511c1f5c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:29.845979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693e389-83ed-41e6-8ce4-57e9aa92908b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:29.846182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 272/539] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:29 2022] GET /identity/v3/role_assignments?user.id=fa2c8be3105a415ead65579dc18ba40d&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:29.846182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18bbc5-e6c8-40f5-ad48-54d0511c1f5c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:29.847344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693e389-83ed-41e6-8ce4-57e9aa92908b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:29.847833 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693e389-83ed-41e6-8ce4-57e9aa92908b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:29.848285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693e389-83ed-41e6-8ce4-57e9aa92908b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:29.849431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693e389-83ed-41e6-8ce4-57e9aa92908b None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=1a9c3f34eaeb4b538528ff9050252dc7) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:29.849921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693e389-83ed-41e6-8ce4-57e9aa92908b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:29.851706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693e389-83ed-41e6-8ce4-57e9aa92908b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:29.865925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 268/540] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:29 2022] GET /identity/v3/role_assignments?user.id=1a9c3f34eaeb4b538528ff9050252dc7&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:29.925595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:29.980385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18bbc5-e6c8-40f5-ad48-54d0511c1f5c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:30.020729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7739c8-f38d-4b39-bbd1-b2e5c5dd7f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:30.021398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7739c8-f38d-4b39-bbd1-b2e5c5dd7f80 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:30.021681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7739c8-f38d-4b39-bbd1-b2e5c5dd7f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:30.021980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7739c8-f38d-4b39-bbd1-b2e5c5dd7f80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:30.022835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7739c8-f38d-4b39-bbd1-b2e5c5dd7f80 None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=710c422b8024403da666ae5054ba1289) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:30.023147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7739c8-f38d-4b39-bbd1-b2e5c5dd7f80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:30.024586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7739c8-f38d-4b39-bbd1-b2e5c5dd7f80 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:30.031338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 273/541] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:29 2022] GET /identity/v3/role_assignments?user.id=710c422b8024403da666ae5054ba1289&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:30.874107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779509f4-c2e8-4f0d-b3e5-72da81d3df89 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:30.875755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779509f4-c2e8-4f0d-b3e5-72da81d3df89 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:30.876247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779509f4-c2e8-4f0d-b3e5-72da81d3df89 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:30.878124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 269/542] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:30.883324 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b393b28-5284-4fe1-9393-61a9a83d5502 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:30.884133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b393b28-5284-4fe1-9393-61a9a83d5502 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:30.884611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b393b28-5284-4fe1-9393-61a9a83d5502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:30.904106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6b393b28-5284-4fe1-9393-61a9a83d5502 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:30.905619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6b393b28-5284-4fe1-9393-61a9a83d5502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:30.995589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 274/543] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:31.037055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779509f4-c2e8-4f0d-b3e5-72da81d3df89 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:31.058267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:31.172677 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779509f4-c2e8-4f0d-b3e5-72da81d3df89 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:31.231244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6e8e8-9aab-4609-b8c7-d416e922dae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:31.232068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6e8e8-9aab-4609-b8c7-d416e922dae7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:31.232351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6e8e8-9aab-4609-b8c7-d416e922dae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:31.232629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6e8e8-9aab-4609-b8c7-d416e922dae7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:31.255325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6e8e8-9aab-4609-b8c7-d416e922dae7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-99a6e8e8-9aab-4609-b8c7-d416e922dae7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:31.275542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:31.276467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:31.277263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 270/544] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:31.283263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b393b28-5284-4fe1-9393-61a9a83d5502 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:31.331778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f1739-2591-4c1e-90bb-4ca326942afd None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:31.332096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f1739-2591-4c1e-90bb-4ca326942afd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:31.332346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f1739-2591-4c1e-90bb-4ca326942afd None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:31.333966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 271/545] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:17:31.339008 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97654425-c817-42c9-b5d3-7569ffcccc6b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:31.339295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97654425-c817-42c9-b5d3-7569ffcccc6b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:31.339576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97654425-c817-42c9-b5d3-7569ffcccc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:31.364582 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-97654425-c817-42c9-b5d3-7569ffcccc6b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:31.365764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-97654425-c817-42c9-b5d3-7569ffcccc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:31.398786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:31.532725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b393b28-5284-4fe1-9393-61a9a83d5502 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:31.537924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 272/546] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 21 06:17:31.597404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97654425-c817-42c9-b5d3-7569ffcccc6b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:31.610568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:31.619181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70a6df-39b7-46fa-9d16-2977660391e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:31.620255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70a6df-39b7-46fa-9d16-2977660391e3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:31.620548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70a6df-39b7-46fa-9d16-2977660391e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:31.620828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70a6df-39b7-46fa-9d16-2977660391e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:31.621598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70a6df-39b7-46fa-9d16-2977660391e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:31.621908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70a6df-39b7-46fa-9d16-2977660391e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:31.624143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70a6df-39b7-46fa-9d16-2977660391e3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:31.636908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 275/547] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:31.642560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70a6df-39b7-46fa-9d16-2977660391e3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:31.852423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97654425-c817-42c9-b5d3-7569ffcccc6b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:31.913695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70a6df-39b7-46fa-9d16-2977660391e3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:31.942308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:31.959135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9983f-d2af-4be6-bdb8-52de44cc2b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:31.959955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9983f-d2af-4be6-bdb8-52de44cc2b19 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:31.960242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9983f-d2af-4be6-bdb8-52de44cc2b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:31.960521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9983f-d2af-4be6-bdb8-52de44cc2b19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:31.994252 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9983f-d2af-4be6-bdb8-52de44cc2b19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-15e9983f-d2af-4be6-bdb8-52de44cc2b19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:32.010284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:32.011208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:32.011998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:32.014709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 273/548] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:32.022909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9983f-d2af-4be6-bdb8-52de44cc2b19 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:32.036669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7e7c7-6e5b-4ffc-b779-73f15ffb9614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:32.037492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7e7c7-6e5b-4ffc-b779-73f15ffb9614 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.037774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7e7c7-6e5b-4ffc-b779-73f15ffb9614 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.038082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7e7c7-6e5b-4ffc-b779-73f15ffb9614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.078961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7e7c7-6e5b-4ffc-b779-73f15ffb9614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:32.080450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7e7c7-6e5b-4ffc-b779-73f15ffb9614 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d4f7e7c7-6e5b-4ffc-b779-73f15ffb9614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:32.116594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:32.117968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:32.118987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:32.118987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:32.118987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:32.118987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:32.118987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 276/549] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 476 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:32.123881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7e7c7-6e5b-4ffc-b779-73f15ffb9614 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:32.195388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:32.269738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9983f-d2af-4be6-bdb8-52de44cc2b19 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:32.315559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7e7c7-6e5b-4ffc-b779-73f15ffb9614 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:32.337251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7647b-7ca2-4dad-9407-40c6474b9499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:32.337956 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7647b-7ca2-4dad-9407-40c6474b9499 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.338248 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7647b-7ca2-4dad-9407-40c6474b9499 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.338532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7647b-7ca2-4dad-9407-40c6474b9499 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.339300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7647b-7ca2-4dad-9407-40c6474b9499 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:32.339613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7647b-7ca2-4dad-9407-40c6474b9499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:32.340998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7647b-7ca2-4dad-9407-40c6474b9499 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:32.370120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 274/550] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:32.371636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7647b-7ca2-4dad-9407-40c6474b9499 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:32.387126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d2ac2-f696-4390-9c09-0d0c2b69de4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:32.387801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d2ac2-f696-4390-9c09-0d0c2b69de4f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.388086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d2ac2-f696-4390-9c09-0d0c2b69de4f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.388364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d2ac2-f696-4390-9c09-0d0c2b69de4f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.389103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d2ac2-f696-4390-9c09-0d0c2b69de4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:32.389413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d2ac2-f696-4390-9c09-0d0c2b69de4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:32.392048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d2ac2-f696-4390-9c09-0d0c2b69de4f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:32.408634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 277/551] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:32.414055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d2ac2-f696-4390-9c09-0d0c2b69de4f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:32.436616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:32.509169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7647b-7ca2-4dad-9407-40c6474b9499 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:32.527608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d2ac2-f696-4390-9c09-0d0c2b69de4f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:32.536947 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:32.548139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62fab6-30af-41b6-90f2-adebe5365e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:32.548932 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62fab6-30af-41b6-90f2-adebe5365e7b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.549215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62fab6-30af-41b6-90f2-adebe5365e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.549495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62fab6-30af-41b6-90f2-adebe5365e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.559897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62fab6-30af-41b6-90f2-adebe5365e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:32.561457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62fab6-30af-41b6-90f2-adebe5365e7b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:32.563437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c485ad-76be-4578-a90d-3ec4082421ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:32.564230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c485ad-76be-4578-a90d-3ec4082421ce None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.564517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c485ad-76be-4578-a90d-3ec4082421ce None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.564800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c485ad-76be-4578-a90d-3ec4082421ce None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.571855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c485ad-76be-4578-a90d-3ec4082421ce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:32.573277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c485ad-76be-4578-a90d-3ec4082421ce None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8b62fab6-30af-41b6-90f2-adebe5365e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:32.580150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:32.581099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:32.582634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:32.582634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:32.582634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:32.582634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:32.582634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 275/552] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:32.582909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277ff6-bc3d-4d5b-8f3d-4620c2f83a2a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.583152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277ff6-bc3d-4d5b-8f3d-4620c2f83a2a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.583412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277ff6-bc3d-4d5b-8f3d-4620c2f83a2a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.584873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 276/553] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:32.590107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0713e-6597-4883-90a0-15e2dc99146a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.590107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0713e-6597-4883-90a0-15e2dc99146a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.590107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0713e-6597-4883-90a0-15e2dc99146a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.590107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 277/554] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-71c485ad-76be-4578-a90d-3ec4082421ce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:32.590582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:32.591401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:32.592373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:32.592373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:32.592373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:32.592373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:32.592373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:32.592373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:32.592373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:32.592373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:32.592373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:17:32.592373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:32.592915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb223052-2fd6-4fb9-b8d0-36d018a8cce6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.592915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb223052-2fd6-4fb9-b8d0-36d018a8cce6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.592915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb223052-2fd6-4fb9-b8d0-36d018a8cce6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.593097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 278/555] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:32.593097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c485ad-76be-4578-a90d-3ec4082421ce None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:32.593215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 278/556] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:32.595440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0dfed-d3f1-4aaa-9b3d-6fb5f9e1dfb5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.596021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0dfed-d3f1-4aaa-9b3d-6fb5f9e1dfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.596538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0dfed-d3f1-4aaa-9b3d-6fb5f9e1dfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.613168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f4e0dfed-d3f1-4aaa-9b3d-6fb5f9e1dfb5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:32.614384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f4e0dfed-d3f1-4aaa-9b3d-6fb5f9e1dfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:32.676671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:32.681761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 279/557] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:17:32.682619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0dfed-d3f1-4aaa-9b3d-6fb5f9e1dfb5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:32.707896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c485ad-76be-4578-a90d-3ec4082421ce None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:32.741907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd1a87-faab-434b-a2bd-5bad3396c689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:32.742590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd1a87-faab-434b-a2bd-5bad3396c689 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.742873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd1a87-faab-434b-a2bd-5bad3396c689 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.743146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd1a87-faab-434b-a2bd-5bad3396c689 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.743866 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd1a87-faab-434b-a2bd-5bad3396c689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:32.744168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd1a87-faab-434b-a2bd-5bad3396c689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:32.745535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd1a87-faab-434b-a2bd-5bad3396c689 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:32.757221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 279/558] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:32.758432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1ac4c-ebeb-4e2c-9349-3041a1027e4e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.758715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1ac4c-ebeb-4e2c-9349-3041a1027e4e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.758992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1ac4c-ebeb-4e2c-9349-3041a1027e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.775849 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e6d1ac4c-ebeb-4e2c-9349-3041a1027e4e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:32.777031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e6d1ac4c-ebeb-4e2c-9349-3041a1027e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:32.781786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0dfed-d3f1-4aaa-9b3d-6fb5f9e1dfb5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:32.789204 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:32.817114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfca13e-8507-4b15-a58e-9f67cc5e2cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:32.817784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfca13e-8507-4b15-a58e-9f67cc5e2cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.818104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfca13e-8507-4b15-a58e-9f67cc5e2cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.818388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfca13e-8507-4b15-a58e-9f67cc5e2cc6 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.819232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfca13e-8507-4b15-a58e-9f67cc5e2cc6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:32.819550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfca13e-8507-4b15-a58e-9f67cc5e2cc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:32.820999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfca13e-8507-4b15-a58e-9f67cc5e2cc6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:32.843152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 280/559] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 06:17:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:32.843152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b64fc-5ff1-4bcb-afa6-40e54dc381cf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.843152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b64fc-5ff1-4bcb-afa6-40e54dc381cf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.843152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b64fc-5ff1-4bcb-afa6-40e54dc381cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:32.848115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 280/560] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:32.848868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1ac4c-ebeb-4e2c-9349-3041a1027e4e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:32.863158 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1c1b64fc-5ff1-4bcb-afa6-40e54dc381cf None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:32.865004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1c1b64fc-5ff1-4bcb-afa6-40e54dc381cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:32.916678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:32.932259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 281/561] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:32.933031 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b64fc-5ff1-4bcb-afa6-40e54dc381cf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:32.955189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1ac4c-ebeb-4e2c-9349-3041a1027e4e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:32.989919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e460d9-f013-4f03-8372-8c7c13c5d05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:32.990735 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e460d9-f013-4f03-8372-8c7c13c5d05e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:32.991024 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e460d9-f013-4f03-8372-8c7c13c5d05e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:32.991304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e460d9-f013-4f03-8372-8c7c13c5d05e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.002576 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e460d9-f013-4f03-8372-8c7c13c5d05e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-55e460d9-f013-4f03-8372-8c7c13c5d05e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:33.010323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:33.011326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 281/562] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:33.012311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e460d9-f013-4f03-8372-8c7c13c5d05e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:33.027558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:33.036418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b64fc-5ff1-4bcb-afa6-40e54dc381cf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:33.071949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5ad34-cff7-4160-a9ba-552caf91d9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:33.072742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5ad34-cff7-4160-a9ba-552caf91d9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:33.073029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5ad34-cff7-4160-a9ba-552caf91d9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:33.073313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5ad34-cff7-4160-a9ba-552caf91d9c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.080664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5ad34-cff7-4160-a9ba-552caf91d9c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:33.082168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5ad34-cff7-4160-a9ba-552caf91d9c2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-93f5ad34-cff7-4160-a9ba-552caf91d9c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:33.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:33.098613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 282/563] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 06:17:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:33.099590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5ad34-cff7-4160-a9ba-552caf91d9c2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:33.120330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e460d9-f013-4f03-8372-8c7c13c5d05e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:33.160013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:33.166686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c990bd-4df1-48fc-9997-534934b9b4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:33.167336 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c990bd-4df1-48fc-9997-534934b9b4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:33.167620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c990bd-4df1-48fc-9997-534934b9b4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:33.167894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c990bd-4df1-48fc-9997-534934b9b4b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.210967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5ad34-cff7-4160-a9ba-552caf91d9c2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:33.236269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c990bd-4df1-48fc-9997-534934b9b4b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:33.237726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c990bd-4df1-48fc-9997-534934b9b4b7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:33.245016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ec02b-5d2b-4a8c-b7ca-b515b7cb9619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:33.245805 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ec02b-5d2b-4a8c-b7ca-b515b7cb9619 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:33.246125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ec02b-5d2b-4a8c-b7ca-b515b7cb9619 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:33.246409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ec02b-5d2b-4a8c-b7ca-b515b7cb9619 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.257081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ec02b-5d2b-4a8c-b7ca-b515b7cb9619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a17ec02b-5d2b-4a8c-b7ca-b515b7cb9619 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:33.264837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:33.265924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 283/564] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:33.267185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ec02b-5d2b-4a8c-b7ca-b515b7cb9619 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:33.282773 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:33.340909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 282/565] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:33.341745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c990bd-4df1-48fc-9997-534934b9b4b7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:33.369407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ec02b-5d2b-4a8c-b7ca-b515b7cb9619 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:33.371318 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:33.402969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54452c6e-3702-4b55-bfee-bd6241f4c35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:33.403758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54452c6e-3702-4b55-bfee-bd6241f4c35b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:33.404047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54452c6e-3702-4b55-bfee-bd6241f4c35b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:33.404330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54452c6e-3702-4b55-bfee-bd6241f4c35b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.415114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54452c6e-3702-4b55-bfee-bd6241f4c35b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-54452c6e-3702-4b55-bfee-bd6241f4c35b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:33.423411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:33.424382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:33.425203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 284/566] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:33.426309 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54452c6e-3702-4b55-bfee-bd6241f4c35b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:33.449239 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c990bd-4df1-48fc-9997-534934b9b4b7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:33.483308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f500d-9f69-4e72-b1cd-ed92d0cb0540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:33.484589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f500d-9f69-4e72-b1cd-ed92d0cb0540 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:33.484876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f500d-9f69-4e72-b1cd-ed92d0cb0540 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:33.485154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f500d-9f69-4e72-b1cd-ed92d0cb0540 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.485891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f500d-9f69-4e72-b1cd-ed92d0cb0540 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:33.486227 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f500d-9f69-4e72-b1cd-ed92d0cb0540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:33.487584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f500d-9f69-4e72-b1cd-ed92d0cb0540 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:33.495599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 283/567] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:33.496374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f500d-9f69-4e72-b1cd-ed92d0cb0540 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:33.498428 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:33.525934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54452c6e-3702-4b55-bfee-bd6241f4c35b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:33.560853 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eae014-a276-4c9b-9a6d-532114d0e598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:33.562186 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eae014-a276-4c9b-9a6d-532114d0e598 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:33.562486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eae014-a276-4c9b-9a6d-532114d0e598 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:33.562846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eae014-a276-4c9b-9a6d-532114d0e598 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.563704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eae014-a276-4c9b-9a6d-532114d0e598 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:33.564013 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eae014-a276-4c9b-9a6d-532114d0e598 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:33.565464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eae014-a276-4c9b-9a6d-532114d0e598 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:33.583121 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 285/568] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:33.583953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eae014-a276-4c9b-9a6d-532114d0e598 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:33.592450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:33.601596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f500d-9f69-4e72-b1cd-ed92d0cb0540 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:33.635567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41105f3-303d-4cfd-920f-9837d6ab0ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:33.636234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41105f3-303d-4cfd-920f-9837d6ab0ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:33.636521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41105f3-303d-4cfd-920f-9837d6ab0ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:33.636799 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41105f3-303d-4cfd-920f-9837d6ab0ea8 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.637551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41105f3-303d-4cfd-920f-9837d6ab0ea8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:33.637863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41105f3-303d-4cfd-920f-9837d6ab0ea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:33.639283 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41105f3-303d-4cfd-920f-9837d6ab0ea8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:33.647944 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 284/569] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:33.648763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41105f3-303d-4cfd-920f-9837d6ab0ea8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:33.683782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eae014-a276-4c9b-9a6d-532114d0e598 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:33.713665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:33.718942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f218ee-afd9-49ff-80a7-c41319b0d9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:33.719740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f218ee-afd9-49ff-80a7-c41319b0d9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:33.720027 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f218ee-afd9-49ff-80a7-c41319b0d9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:33.720303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f218ee-afd9-49ff-80a7-c41319b0d9f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.730369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f218ee-afd9-49ff-80a7-c41319b0d9f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:33.731811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f218ee-afd9-49ff-80a7-c41319b0d9f9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:33.751030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41105f3-303d-4cfd-920f-9837d6ab0ea8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c7f218ee-afd9-49ff-80a7-c41319b0d9f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:33.751696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:33.752656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:33.753540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:33.753540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:33.753540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:33.753540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:33.753540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 286/570] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:33.754087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f218ee-afd9-49ff-80a7-c41319b0d9f9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:33.790624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b6477-4073-47e0-8b96-fc41b691f1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:33.791428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b6477-4073-47e0-8b96-fc41b691f1ae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:33.791753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b6477-4073-47e0-8b96-fc41b691f1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:33.792220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b6477-4073-47e0-8b96-fc41b691f1ae None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.793056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b6477-4073-47e0-8b96-fc41b691f1ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:33.793440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b6477-4073-47e0-8b96-fc41b691f1ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:33.794848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b6477-4073-47e0-8b96-fc41b691f1ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:33.803828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 285/571] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:33.804632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b6477-4073-47e0-8b96-fc41b691f1ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:33.823291 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:33.866386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f218ee-afd9-49ff-80a7-c41319b0d9f9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:33.900364 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a4d5-43d4-4b1f-93da-90429da661ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:33.901159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a4d5-43d4-4b1f-93da-90429da661ba None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:33.901440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a4d5-43d4-4b1f-93da-90429da661ba None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:33.901725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a4d5-43d4-4b1f-93da-90429da661ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.904525 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b6477-4073-47e0-8b96-fc41b691f1ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:33.912810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a4d5-43d4-4b1f-93da-90429da661ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:33.914295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a4d5-43d4-4b1f-93da-90429da661ba None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fc47a4d5-43d4-4b1f-93da-90429da661ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:33.931670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:33.932767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:33.933673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:33.933673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:33.933673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:33.933673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:33.933673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 287/572] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:33.933983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a4d5-43d4-4b1f-93da-90429da661ba None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:33.941060 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af145e4-175e-4271-b5b4-0eaf57711864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:33.941857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af145e4-175e-4271-b5b4-0eaf57711864 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:33.942169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af145e4-175e-4271-b5b4-0eaf57711864 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:33.942445 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af145e4-175e-4271-b5b4-0eaf57711864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:33.955916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af145e4-175e-4271-b5b4-0eaf57711864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:33.957379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af145e4-175e-4271-b5b4-0eaf57711864 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:33.971196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0af145e4-175e-4271-b5b4-0eaf57711864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:33.975090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:33.976049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:33.976921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:33.976921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:33.976921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:33.976921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:33.976921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 286/573] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:33.981655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af145e4-175e-4271-b5b4-0eaf57711864 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.037989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a4d5-43d4-4b1f-93da-90429da661ba None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.046965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:34.076972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d00fc7-c5a1-4eb1-95d4-48770a6a7cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.078464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d00fc7-c5a1-4eb1-95d4-48770a6a7cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.078747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d00fc7-c5a1-4eb1-95d4-48770a6a7cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.079026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d00fc7-c5a1-4eb1-95d4-48770a6a7cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.086492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af145e4-175e-4271-b5b4-0eaf57711864 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.089346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d00fc7-c5a1-4eb1-95d4-48770a6a7cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.090824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d00fc7-c5a1-4eb1-95d4-48770a6a7cf3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-33d00fc7-c5a1-4eb1-95d4-48770a6a7cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:34.107551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:34.108500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:34.109562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:34.109562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:34.109562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:34.109562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:34.109562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 288/574] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:34.109562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d00fc7-c5a1-4eb1-95d4-48770a6a7cf3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.121637 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a88da-2afa-4a81-8197-c30ad01f2745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.122332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a88da-2afa-4a81-8197-c30ad01f2745 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.122618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a88da-2afa-4a81-8197-c30ad01f2745 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.122899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a88da-2afa-4a81-8197-c30ad01f2745 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.123628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a88da-2afa-4a81-8197-c30ad01f2745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:34.124492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a88da-2afa-4a81-8197-c30ad01f2745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.125861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a88da-2afa-4a81-8197-c30ad01f2745 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.137645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 287/575] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:34.138461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a88da-2afa-4a81-8197-c30ad01f2745 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.197898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:34.210733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d00fc7-c5a1-4eb1-95d4-48770a6a7cf3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.240308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a88da-2afa-4a81-8197-c30ad01f2745 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.242657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5f0a0-482e-4cc5-90f1-0fc95f523506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.243446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5f0a0-482e-4cc5-90f1-0fc95f523506 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.243728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5f0a0-482e-4cc5-90f1-0fc95f523506 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.244005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5f0a0-482e-4cc5-90f1-0fc95f523506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.254790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5f0a0-482e-4cc5-90f1-0fc95f523506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.256243 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5f0a0-482e-4cc5-90f1-0fc95f523506 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.266831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-59e5f0a0-482e-4cc5-90f1-0fc95f523506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:34.273787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:34.275407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:34.276363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:34.276363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:34.276363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:34.276363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:34.276597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bec2c1-448f-4f04-9880-65ef7729b50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.276685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 289/576] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:34.276805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bec2c1-448f-4f04-9880-65ef7729b50f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.276805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bec2c1-448f-4f04-9880-65ef7729b50f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.276942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5f0a0-482e-4cc5-90f1-0fc95f523506 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.277014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bec2c1-448f-4f04-9880-65ef7729b50f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.277014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bec2c1-448f-4f04-9880-65ef7729b50f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:34.277207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bec2c1-448f-4f04-9880-65ef7729b50f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.278599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bec2c1-448f-4f04-9880-65ef7729b50f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.295483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 288/577] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:34.295483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bec2c1-448f-4f04-9880-65ef7729b50f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.378586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5f0a0-482e-4cc5-90f1-0fc95f523506 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.391181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bec2c1-448f-4f04-9880-65ef7729b50f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.412148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dca648-b6e4-4f38-8c7f-2170d2d467ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.412804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dca648-b6e4-4f38-8c7f-2170d2d467ca None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.413087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dca648-b6e4-4f38-8c7f-2170d2d467ca None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.413360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dca648-b6e4-4f38-8c7f-2170d2d467ca None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.414111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dca648-b6e4-4f38-8c7f-2170d2d467ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:34.414417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dca648-b6e4-4f38-8c7f-2170d2d467ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.415772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dca648-b6e4-4f38-8c7f-2170d2d467ca None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.424187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5342e7e-4e10-4130-9909-22e6702f8589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.424852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5342e7e-4e10-4130-9909-22e6702f8589 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.425137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5342e7e-4e10-4130-9909-22e6702f8589 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.425419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5342e7e-4e10-4130-9909-22e6702f8589 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.426001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:34.426181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5342e7e-4e10-4130-9909-22e6702f8589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:34.426491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5342e7e-4e10-4130-9909-22e6702f8589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.428166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5342e7e-4e10-4130-9909-22e6702f8589 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.429678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 290/578] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:34.430413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dca648-b6e4-4f38-8c7f-2170d2d467ca None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.441953 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 289/579] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:34.442779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5342e7e-4e10-4130-9909-22e6702f8589 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.491025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:34.536505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dca648-b6e4-4f38-8c7f-2170d2d467ca None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.542709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5342e7e-4e10-4130-9909-22e6702f8589 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.570746 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eca381-679f-45b9-a95e-e06233a51e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.571463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eca381-679f-45b9-a95e-e06233a51e78 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.571748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eca381-679f-45b9-a95e-e06233a51e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.572020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eca381-679f-45b9-a95e-e06233a51e78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.576961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5fa3f-f80b-4a7e-a397-f981485380ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.577637 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5fa3f-f80b-4a7e-a397-f981485380ae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.577917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5fa3f-f80b-4a7e-a397-f981485380ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.578220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5fa3f-f80b-4a7e-a397-f981485380ae None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.578644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eca381-679f-45b9-a95e-e06233a51e78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.578943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5fa3f-f80b-4a7e-a397-f981485380ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:34.579253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5fa3f-f80b-4a7e-a397-f981485380ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.580077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eca381-679f-45b9-a95e-e06233a51e78 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.580633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5fa3f-f80b-4a7e-a397-f981485380ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.591833 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 290/580] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:34.592456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5fa3f-f80b-4a7e-a397-f981485380ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-56eca381-679f-45b9-a95e-e06233a51e78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:34.593905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:34.594757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:34.595656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:34.595656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:34.595656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:34.595656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:34.595656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:34.595656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:34.595656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:34.595656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:34.595656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 291/581] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:34.596145 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eca381-679f-45b9-a95e-e06233a51e78 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.654668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:34.695214 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eca381-679f-45b9-a95e-e06233a51e78 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.696437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5fa3f-f80b-4a7e-a397-f981485380ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.705079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:34.727786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395c421-eedf-430f-87cf-01981ef07db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.728571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395c421-eedf-430f-87cf-01981ef07db1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.728858 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395c421-eedf-430f-87cf-01981ef07db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.729132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395c421-eedf-430f-87cf-01981ef07db1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.730086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda25b0d-7657-4447-8e69-71bb412439fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.730876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda25b0d-7657-4447-8e69-71bb412439fb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.731159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda25b0d-7657-4447-8e69-71bb412439fb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.731434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda25b0d-7657-4447-8e69-71bb412439fb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.736477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395c421-eedf-430f-87cf-01981ef07db1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.737913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395c421-eedf-430f-87cf-01981ef07db1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.739340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda25b0d-7657-4447-8e69-71bb412439fb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.740759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda25b0d-7657-4447-8e69-71bb412439fb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3395c421-eedf-430f-87cf-01981ef07db1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:34.752585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:34.753616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:34.754603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:34.754603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:34.754603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:34.754603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:34.754603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:34.754603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:34.754603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:34.754603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:34.754603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 292/582] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:34.755093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395c421-eedf-430f-87cf-01981ef07db1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fda25b0d-7657-4447-8e69-71bb412439fb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:34.755202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:34.756050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:34.756983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:34.756983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:34.756983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:34.756983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:34.756983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:34.756983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:34.756983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:34.756983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:34.756983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 06:17:34.756983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:34.756983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 291/583] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:34.757569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda25b0d-7657-4447-8e69-71bb412439fb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.859931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395c421-eedf-430f-87cf-01981ef07db1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.860206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda25b0d-7657-4447-8e69-71bb412439fb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:34.880311 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:34.893170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2b91a-305b-43e0-a4f7-006d02abda15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.893986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b91a-305b-43e0-a4f7-006d02abda15 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.894271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b91a-305b-43e0-a4f7-006d02abda15 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.894390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f99894-878e-4596-be94-5a418ad7c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:34.894548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b91a-305b-43e0-a4f7-006d02abda15 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.895102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99894-878e-4596-be94-5a418ad7c9cf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:34.895387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99894-878e-4596-be94-5a418ad7c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:34.895670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99894-878e-4596-be94-5a418ad7c9cf None admin] PATH_INFO: `/v3/users/glance` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:34.902188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2b91a-305b-43e0-a4f7-006d02abda15 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.902960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f99894-878e-4596-be94-5a418ad7c9cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:34.903623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2b91a-305b-43e0-a4f7-006d02abda15 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.904379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f99894-878e-4596-be94-5a418ad7c9cf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-95e2b91a-305b-43e0-a4f7-006d02abda15 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:34.918111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:34.919197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:34.926589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:34.926589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:34.926589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:34.926589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:34.926589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:34.926589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:34.926589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:34.926589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:34.926589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 06:17:34.926589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b2f99894-878e-4596-be94-5a418ad7c9cf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:34.927230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:34.928239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:34.929357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:34.929357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:34.929357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:34.929357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:34.929357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:34.929357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:34.929357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:34.929357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:34.929357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 06:17:34.929357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:34.930007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 292/584] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:34.930007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2b91a-305b-43e0-a4f7-006d02abda15 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.930170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 293/585] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:34.930170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f99894-878e-4596-be94-5a418ad7c9cf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:34.954296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:35.021001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f99894-878e-4596-be94-5a418ad7c9cf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.026413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2b91a-305b-43e0-a4f7-006d02abda15 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.058058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d340b80-d21f-46ee-bff2-5bf21478e1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:35.058730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d340b80-d21f-46ee-bff2-5bf21478e1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:35.059017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d340b80-d21f-46ee-bff2-5bf21478e1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:35.059311 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d340b80-d21f-46ee-bff2-5bf21478e1d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:35.060089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d340b80-d21f-46ee-bff2-5bf21478e1d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:35.060399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d340b80-d21f-46ee-bff2-5bf21478e1d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:35.062400 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d340b80-d21f-46ee-bff2-5bf21478e1d9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:35.065015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23a995-4bdd-43af-838e-6745cfd65249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:35.065681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23a995-4bdd-43af-838e-6745cfd65249 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:35.066010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23a995-4bdd-43af-838e-6745cfd65249 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:35.066295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23a995-4bdd-43af-838e-6745cfd65249 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:35.067090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23a995-4bdd-43af-838e-6745cfd65249 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:35.067402 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23a995-4bdd-43af-838e-6745cfd65249 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:35.068829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23a995-4bdd-43af-838e-6745cfd65249 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:35.073970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 294/586] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:35.074782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d340b80-d21f-46ee-bff2-5bf21478e1d9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:35.082621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 293/587] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:35.083438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23a995-4bdd-43af-838e-6745cfd65249 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:35.116511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:35.168479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:35.175459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d340b80-d21f-46ee-bff2-5bf21478e1d9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.180496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23a995-4bdd-43af-838e-6745cfd65249 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.209385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaed7d-443a-4a73-9249-6c448d717e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:35.210086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaed7d-443a-4a73-9249-6c448d717e75 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:35.210373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaed7d-443a-4a73-9249-6c448d717e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:35.210658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaed7d-443a-4a73-9249-6c448d717e75 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:35.211508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaed7d-443a-4a73-9249-6c448d717e75 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:35.211822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaed7d-443a-4a73-9249-6c448d717e75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:35.213248 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaed7d-443a-4a73-9249-6c448d717e75 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:35.214436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c9631-7aca-4ce5-8fe4-a08ad239bc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:35.215097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c9631-7aca-4ce5-8fe4-a08ad239bc43 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:35.215378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c9631-7aca-4ce5-8fe4-a08ad239bc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:35.215652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c9631-7aca-4ce5-8fe4-a08ad239bc43 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:35.216485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c9631-7aca-4ce5-8fe4-a08ad239bc43 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:35.216792 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c9631-7aca-4ce5-8fe4-a08ad239bc43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:35.218264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c9631-7aca-4ce5-8fe4-a08ad239bc43 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:35.230436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 295/588] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 06:17:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:35.231246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaed7d-443a-4a73-9249-6c448d717e75 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:35.254347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 294/589] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Jul 21 06:17:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:35.255142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c9631-7aca-4ce5-8fe4-a08ad239bc43 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:35.331582 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:35.335362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaed7d-443a-4a73-9249-6c448d717e75 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.354275 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c9631-7aca-4ce5-8fe4-a08ad239bc43 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.369025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf864a-3757-4d43-b6fd-4f8d64988f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:35.369635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf864a-3757-4d43-b6fd-4f8d64988f54 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:35.369916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf864a-3757-4d43-b6fd-4f8d64988f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:35.370285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf864a-3757-4d43-b6fd-4f8d64988f54 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:35.371114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf864a-3757-4d43-b6fd-4f8d64988f54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:35.371416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf864a-3757-4d43-b6fd-4f8d64988f54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:35.372818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf864a-3757-4d43-b6fd-4f8d64988f54 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:35.390697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 296/590] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 06:17:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:35.390917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a8553-24db-4cc8-949c-54042d93f4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:35.391607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf864a-3757-4d43-b6fd-4f8d64988f54 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:35.391910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a8553-24db-4cc8-949c-54042d93f4df None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:35.392127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a8553-24db-4cc8-949c-54042d93f4df None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:35.392404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a8553-24db-4cc8-949c-54042d93f4df None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/058f099e6f7545a1b2734dbe5afd2027/roles/7541347b464141a78014fd57dccc765b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:35.407453 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a8553-24db-4cc8-949c-54042d93f4df None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=058f099e6f7545a1b2734dbe5afd2027, role_id=7541347b464141a78014fd57dccc765b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:35.408920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a8553-24db-4cc8-949c-54042d93f4df None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:35.411896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:35.424465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 295/591] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:17:35 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/058f099e6f7545a1b2734dbe5afd2027/roles/7541347b464141a78014fd57dccc765b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:17:35.425247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a8553-24db-4cc8-949c-54042d93f4df None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:35.503997 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf864a-3757-4d43-b6fd-4f8d64988f54 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.540609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da66ffc-4033-4cc7-a245-be1f3fc136bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:35.541285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da66ffc-4033-4cc7-a245-be1f3fc136bc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:35.541569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da66ffc-4033-4cc7-a245-be1f3fc136bc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:35.541844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da66ffc-4033-4cc7-a245-be1f3fc136bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:35.542749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da66ffc-4033-4cc7-a245-be1f3fc136bc None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=94c8eb61f2ca48288cba81d280a30747) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:35.543066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da66ffc-4033-4cc7-a245-be1f3fc136bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:35.544571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da66ffc-4033-4cc7-a245-be1f3fc136bc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:35.548003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a8553-24db-4cc8-949c-54042d93f4df None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.560581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 297/592] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:35 2022] GET /identity/v3/role_assignments?user.id=94c8eb61f2ca48288cba81d280a30747&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:35.560581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da66ffc-4033-4cc7-a245-be1f3fc136bc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:35.579273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:35.600900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771441b-2ef6-4138-bac9-d69098d191e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:35.601714 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771441b-2ef6-4138-bac9-d69098d191e1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:35.602090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771441b-2ef6-4138-bac9-d69098d191e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:35.602377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771441b-2ef6-4138-bac9-d69098d191e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:35.620574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771441b-2ef6-4138-bac9-d69098d191e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:35.622993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771441b-2ef6-4138-bac9-d69098d191e1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f771441b-2ef6-4138-bac9-d69098d191e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:35.642610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:35.643578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:35.644454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:35.644454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:35.644454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:35.644454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:35.648787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 296/593] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:35.650387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771441b-2ef6-4138-bac9-d69098d191e1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:35.700805 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da66ffc-4033-4cc7-a245-be1f3fc136bc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.709196 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:35.746650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749e6ef-67ae-442c-8982-146c310da643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:35.747994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749e6ef-67ae-442c-8982-146c310da643 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:35.748432 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749e6ef-67ae-442c-8982-146c310da643 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:35.749557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749e6ef-67ae-442c-8982-146c310da643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:35.764918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749e6ef-67ae-442c-8982-146c310da643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:35.767233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749e6ef-67ae-442c-8982-146c310da643 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:35.769557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771441b-2ef6-4138-bac9-d69098d191e1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5749e6ef-67ae-442c-8982-146c310da643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:35.790392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:35.791691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:35.792794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:35.792794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:35.792794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:35.792794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:35.794992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 298/594] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:35.796715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749e6ef-67ae-442c-8982-146c310da643 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:35.808253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1367119-b80c-42b4-bcc2-4f3657b72bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:35.809111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1367119-b80c-42b4-bcc2-4f3657b72bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:35.809500 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1367119-b80c-42b4-bcc2-4f3657b72bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:35.809884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1367119-b80c-42b4-bcc2-4f3657b72bd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:35.823000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1367119-b80c-42b4-bcc2-4f3657b72bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:35.824923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1367119-b80c-42b4-bcc2-4f3657b72bd0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f1367119-b80c-42b4-bcc2-4f3657b72bd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:35.847317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:35.848330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:35.849195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:35.849195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:35.849195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:35.849195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:35.849195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 297/595] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:35.850416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1367119-b80c-42b4-bcc2-4f3657b72bd0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:35.887529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:35.932020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749e6ef-67ae-442c-8982-146c310da643 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.970620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2754c-60b2-43b9-abf8-6e2d62672daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:35.971404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2754c-60b2-43b9-abf8-6e2d62672daa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:35.972033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2754c-60b2-43b9-abf8-6e2d62672daa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:35.972147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2754c-60b2-43b9-abf8-6e2d62672daa None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:35.973030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2754c-60b2-43b9-abf8-6e2d62672daa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:35.973412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2754c-60b2-43b9-abf8-6e2d62672daa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:35.974975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2754c-60b2-43b9-abf8-6e2d62672daa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:35.975927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:35.980088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1367119-b80c-42b4-bcc2-4f3657b72bd0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:35.991481 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 299/596] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:35.992247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2754c-60b2-43b9-abf8-6e2d62672daa None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:36.031538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccf422-87d9-4cf5-afe3-6ef00772412f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:36.032564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccf422-87d9-4cf5-afe3-6ef00772412f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:36.033028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccf422-87d9-4cf5-afe3-6ef00772412f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:36.033510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccf422-87d9-4cf5-afe3-6ef00772412f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:36.034481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccf422-87d9-4cf5-afe3-6ef00772412f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:36.035056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccf422-87d9-4cf5-afe3-6ef00772412f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:36.036639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccf422-87d9-4cf5-afe3-6ef00772412f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:36.052247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 298/597] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:36.053537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccf422-87d9-4cf5-afe3-6ef00772412f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:36.103465 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2754c-60b2-43b9-abf8-6e2d62672daa None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:36.140925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee11376-a58b-489a-a907-34cb72c30491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:36.141520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee11376-a58b-489a-a907-34cb72c30491 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:36.141859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee11376-a58b-489a-a907-34cb72c30491 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:36.142114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee11376-a58b-489a-a907-34cb72c30491 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:36.142847 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee11376-a58b-489a-a907-34cb72c30491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:36.143156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee11376-a58b-489a-a907-34cb72c30491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:36.144620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee11376-a58b-489a-a907-34cb72c30491 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:36.157482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 300/598] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:36.157865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:36.162034 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee11376-a58b-489a-a907-34cb72c30491 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:36.166147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccf422-87d9-4cf5-afe3-6ef00772412f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:36.203655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd168b-9d8f-4b72-bc8b-28b4c89eb01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:36.204755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd168b-9d8f-4b72-bc8b-28b4c89eb01a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:36.205213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd168b-9d8f-4b72-bc8b-28b4c89eb01a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:36.205736 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd168b-9d8f-4b72-bc8b-28b4c89eb01a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:36.213572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd168b-9d8f-4b72-bc8b-28b4c89eb01a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:36.215427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd168b-9d8f-4b72-bc8b-28b4c89eb01a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-61bd168b-9d8f-4b72-bc8b-28b4c89eb01a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:36.230451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:36.231479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:36.232350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:36.232350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:36.232350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:36.232350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:36.232350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:36.232350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:36.232350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:36.232350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:36.239830 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 299/599] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:36.240886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd168b-9d8f-4b72-bc8b-28b4c89eb01a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:36.264613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:36.285976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee11376-a58b-489a-a907-34cb72c30491 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:36.329200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa6ee8-5fa6-4466-8adf-c890e5f2755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:36.330355 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa6ee8-5fa6-4466-8adf-c890e5f2755f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:36.330756 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa6ee8-5fa6-4466-8adf-c890e5f2755f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:36.331405 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa6ee8-5fa6-4466-8adf-c890e5f2755f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:36.339627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa6ee8-5fa6-4466-8adf-c890e5f2755f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:36.341578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa6ee8-5fa6-4466-8adf-c890e5f2755f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:36.351069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd168b-9d8f-4b72-bc8b-28b4c89eb01a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-feaa6ee8-5fa6-4466-8adf-c890e5f2755f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:36.360583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:36.361511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:36.362403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:36.362403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:36.362403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:36.362403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:36.362403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:36.362403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:36.362403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:36.362403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:36.362403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 301/600] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:36.363344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa6ee8-5fa6-4466-8adf-c890e5f2755f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:36.395342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daee7e-f524-461d-a5b3-65f60df85de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:36.396471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daee7e-f524-461d-a5b3-65f60df85de9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:36.396953 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daee7e-f524-461d-a5b3-65f60df85de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:36.397427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daee7e-f524-461d-a5b3-65f60df85de9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:36.404920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7daee7e-f524-461d-a5b3-65f60df85de9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:36.406742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7daee7e-f524-461d-a5b3-65f60df85de9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e7daee7e-f524-461d-a5b3-65f60df85de9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:36.421615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:36.422532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:36.423392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:36.423392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:36.423392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:36.423392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:36.423392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:36.423392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:36.423392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:36.423392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:36.425801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 300/601] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:36.426003 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:36.427359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daee7e-f524-461d-a5b3-65f60df85de9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:36.493464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa6ee8-5fa6-4466-8adf-c890e5f2755f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:36.529759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:36.530297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b456c-a1ad-4998-9c5b-9069bb94aa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:36.531398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b456c-a1ad-4998-9c5b-9069bb94aa5e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:36.531687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b456c-a1ad-4998-9c5b-9069bb94aa5e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:36.531970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b456c-a1ad-4998-9c5b-9069bb94aa5e None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:36.533226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b456c-a1ad-4998-9c5b-9069bb94aa5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:36.533540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b456c-a1ad-4998-9c5b-9069bb94aa5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:36.535261 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b456c-a1ad-4998-9c5b-9069bb94aa5e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:36.549690 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 302/602] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:36.554451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b456c-a1ad-4998-9c5b-9069bb94aa5e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:36.562361 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daee7e-f524-461d-a5b3-65f60df85de9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:36.606737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78f529-682d-4eaf-9742-3f063701ef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:36.607859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78f529-682d-4eaf-9742-3f063701ef98 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:36.609448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78f529-682d-4eaf-9742-3f063701ef98 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:36.609971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78f529-682d-4eaf-9742-3f063701ef98 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:36.611397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78f529-682d-4eaf-9742-3f063701ef98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:36.612232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78f529-682d-4eaf-9742-3f063701ef98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:36.614599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78f529-682d-4eaf-9742-3f063701ef98 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:36.631815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 301/603] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:36.633408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78f529-682d-4eaf-9742-3f063701ef98 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:36.680700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b456c-a1ad-4998-9c5b-9069bb94aa5e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:36.698366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:36.731981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45cd1b-d966-40be-8d14-045df3e6cb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:36.732945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45cd1b-d966-40be-8d14-045df3e6cb09 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:36.733240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45cd1b-d966-40be-8d14-045df3e6cb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:36.733526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45cd1b-d966-40be-8d14-045df3e6cb09 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:36.734864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a45cd1b-d966-40be-8d14-045df3e6cb09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:36.735191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a45cd1b-d966-40be-8d14-045df3e6cb09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:36.736975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a45cd1b-d966-40be-8d14-045df3e6cb09 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:36.755816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78f529-682d-4eaf-9742-3f063701ef98 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:36.758192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 303/604] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:36.759268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45cd1b-d966-40be-8d14-045df3e6cb09 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:36.798334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f9a6-cd7b-4d27-94e2-7d045684b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:36.799831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f9a6-cd7b-4d27-94e2-7d045684b8eb None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:36.799831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f9a6-cd7b-4d27-94e2-7d045684b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:36.800053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f9a6-cd7b-4d27-94e2-7d045684b8eb None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/710c422b8024403da666ae5054ba1289/roles/7541347b464141a78014fd57dccc765b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:36.812123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:36.817062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1f9a6-cd7b-4d27-94e2-7d045684b8eb None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=710c422b8024403da666ae5054ba1289, role_id=7541347b464141a78014fd57dccc765b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:36.818566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1f9a6-cd7b-4d27-94e2-7d045684b8eb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:36.841966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 302/605] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:17:36 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/710c422b8024403da666ae5054ba1289/roles/7541347b464141a78014fd57dccc765b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:17:36.841966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f9a6-cd7b-4d27-94e2-7d045684b8eb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:36.901114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45cd1b-d966-40be-8d14-045df3e6cb09 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:36.964070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f9a6-cd7b-4d27-94e2-7d045684b8eb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:37.002797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4dff1-728a-42fd-a383-98ef141b06e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:37.004241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4dff1-728a-42fd-a383-98ef141b06e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:37.004721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4dff1-728a-42fd-a383-98ef141b06e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:37.005323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4dff1-728a-42fd-a383-98ef141b06e5 None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/1a9c3f34eaeb4b538528ff9050252dc7/roles/7541347b464141a78014fd57dccc765b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:37.005661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec30bb3-c6dc-4ee1-9d65-ec02afd596e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:37.006595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30bb3-c6dc-4ee1-9d65-ec02afd596e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:37.006963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30bb3-c6dc-4ee1-9d65-ec02afd596e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:37.007266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30bb3-c6dc-4ee1-9d65-ec02afd596e6 None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/fa2c8be3105a415ead65579dc18ba40d/roles/7541347b464141a78014fd57dccc765b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:37.027084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec30bb3-c6dc-4ee1-9d65-ec02afd596e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=fa2c8be3105a415ead65579dc18ba40d, role_id=7541347b464141a78014fd57dccc765b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:37.029015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec30bb3-c6dc-4ee1-9d65-ec02afd596e6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:37.038908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c4dff1-728a-42fd-a383-98ef141b06e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=1a9c3f34eaeb4b538528ff9050252dc7, role_id=7541347b464141a78014fd57dccc765b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:37.040398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c4dff1-728a-42fd-a383-98ef141b06e5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:37.058465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 303/606] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:17:36 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/fa2c8be3105a415ead65579dc18ba40d/roles/7541347b464141a78014fd57dccc765b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:17:37.094201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 304/607] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:17:36 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/1a9c3f34eaeb4b538528ff9050252dc7/roles/7541347b464141a78014fd57dccc765b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:17:37.590192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174621a2-bc0f-44ac-86a6-9c95a2c3cccc None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:37.590526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174621a2-bc0f-44ac-86a6-9c95a2c3cccc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:37.590757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174621a2-bc0f-44ac-86a6-9c95a2c3cccc None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:37.592489 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 304/608] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:37.597867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243a779-a9e1-4971-ab6e-610a152b3014 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:37.598337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243a779-a9e1-4971-ab6e-610a152b3014 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:37.598512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243a779-a9e1-4971-ab6e-610a152b3014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:37.616159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f243a779-a9e1-4971-ab6e-610a152b3014 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:37.617359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f243a779-a9e1-4971-ab6e-610a152b3014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:37.635971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:37.707358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 305/609] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:37.743295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174621a2-bc0f-44ac-86a6-9c95a2c3cccc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:37.803527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:37.880330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174621a2-bc0f-44ac-86a6-9c95a2c3cccc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:37.900688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeb3d1-a08c-40ac-8ada-87acef764822 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:37.900976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeb3d1-a08c-40ac-8ada-87acef764822 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:37.901257 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeb3d1-a08c-40ac-8ada-87acef764822 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:37.904274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 306/610] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:37.914523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92aa9-52e7-4170-b3c5-b7323a19ca15 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:37.914785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92aa9-52e7-4170-b3c5-b7323a19ca15 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:37.915065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92aa9-52e7-4170-b3c5-b7323a19ca15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:37.931736 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731ec44-0d0d-45d5-ab51-a504e30a13c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:37.932865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731ec44-0d0d-45d5-ab51-a504e30a13c3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:37.933350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731ec44-0d0d-45d5-ab51-a504e30a13c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:37.933894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731ec44-0d0d-45d5-ab51-a504e30a13c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:37.945307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731ec44-0d0d-45d5-ab51-a504e30a13c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:37.946930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1de92aa9-52e7-4170-b3c5-b7323a19ca15 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:37.948119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1de92aa9-52e7-4170-b3c5-b7323a19ca15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6731ec44-0d0d-45d5-ab51-a504e30a13c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:37.953917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:37.954829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:37.955647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:37.958319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 305/611] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:37.963769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731ec44-0d0d-45d5-ab51-a504e30a13c3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:38.096577 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 307/612] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:38.121519 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de92aa9-52e7-4170-b3c5-b7323a19ca15 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:38.139543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:38.148658 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731ec44-0d0d-45d5-ab51-a504e30a13c3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:38.200625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625ad7c-67c2-442e-a884-f32ed6c50877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:38.201295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625ad7c-67c2-442e-a884-f32ed6c50877 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:38.202172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625ad7c-67c2-442e-a884-f32ed6c50877 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:38.203491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625ad7c-67c2-442e-a884-f32ed6c50877 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:38.204503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625ad7c-67c2-442e-a884-f32ed6c50877 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:38.204816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625ad7c-67c2-442e-a884-f32ed6c50877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:38.206209 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625ad7c-67c2-442e-a884-f32ed6c50877 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:38.223245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 306/613] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:38.228838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625ad7c-67c2-442e-a884-f32ed6c50877 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:38.257326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:38.296416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de92aa9-52e7-4170-b3c5-b7323a19ca15 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:38.348150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eadcf0-c754-4594-a6f0-333e82fbeb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:38.349421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eadcf0-c754-4594-a6f0-333e82fbeb66 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:38.350051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eadcf0-c754-4594-a6f0-333e82fbeb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:38.350552 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eadcf0-c754-4594-a6f0-333e82fbeb66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:38.369067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625ad7c-67c2-442e-a884-f32ed6c50877 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:38.371340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eadcf0-c754-4594-a6f0-333e82fbeb66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e1eadcf0-c754-4594-a6f0-333e82fbeb66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:38.379581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:38.380567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:38.381441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:38.383964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 308/614] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:38.390431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eadcf0-c754-4594-a6f0-333e82fbeb66 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:38.415082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8747b-755e-4600-9d45-7a989afd1d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:38.415878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8747b-755e-4600-9d45-7a989afd1d5e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:38.416208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8747b-755e-4600-9d45-7a989afd1d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:38.416481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8747b-755e-4600-9d45-7a989afd1d5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:38.438844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8747b-755e-4600-9d45-7a989afd1d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:38.440659 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8747b-755e-4600-9d45-7a989afd1d5e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b0e8747b-755e-4600-9d45-7a989afd1d5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:38.481841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:38.482883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:38.483937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:38.483937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:38.483937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:38.483937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:38.494445 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 307/615] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:38.500046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8747b-755e-4600-9d45-7a989afd1d5e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:38.522980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eadcf0-c754-4594-a6f0-333e82fbeb66 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:38.542296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:38.569504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23badbb8-c79e-4541-ae17-5e49a0e5f2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:38.570207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23badbb8-c79e-4541-ae17-5e49a0e5f2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:38.570493 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23badbb8-c79e-4541-ae17-5e49a0e5f2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:38.570771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23badbb8-c79e-4541-ae17-5e49a0e5f2d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:38.571526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23badbb8-c79e-4541-ae17-5e49a0e5f2d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:38.571832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23badbb8-c79e-4541-ae17-5e49a0e5f2d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:38.573211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23badbb8-c79e-4541-ae17-5e49a0e5f2d8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:38.582692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 309/616] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:38.588097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23badbb8-c79e-4541-ae17-5e49a0e5f2d8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:38.595726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:38.650217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8747b-755e-4600-9d45-7a989afd1d5e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:38.702578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5014754-800d-4dde-a5a5-0347fc417cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:38.703250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014754-800d-4dde-a5a5-0347fc417cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:38.703538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014754-800d-4dde-a5a5-0347fc417cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:38.703819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014754-800d-4dde-a5a5-0347fc417cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:38.704662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5014754-800d-4dde-a5a5-0347fc417cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:38.704978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5014754-800d-4dde-a5a5-0347fc417cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:38.706510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5014754-800d-4dde-a5a5-0347fc417cd9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:38.719464 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 308/617] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:38.727020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5014754-800d-4dde-a5a5-0347fc417cd9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:38.748115 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23badbb8-c79e-4541-ae17-5e49a0e5f2d8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:38.801609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75082c-c271-448e-be6c-e84f962fa733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:38.802440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75082c-c271-448e-be6c-e84f962fa733 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:38.806610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75082c-c271-448e-be6c-e84f962fa733 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:38.806900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75082c-c271-448e-be6c-e84f962fa733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:38.824188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75082c-c271-448e-be6c-e84f962fa733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:38.828189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75082c-c271-448e-be6c-e84f962fa733 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ef75082c-c271-448e-be6c-e84f962fa733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:38.859611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:38.860568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:38.861436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:38.861436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:38.861436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:38.861436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:38.862200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 310/618] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:38.867874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75082c-c271-448e-be6c-e84f962fa733 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:38.876023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:38.921655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5014754-800d-4dde-a5a5-0347fc417cd9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:38.971434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:38.993671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ec87a-c50b-4c5b-a842-9b1d5c036005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:38.994505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ec87a-c50b-4c5b-a842-9b1d5c036005 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:38.994793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ec87a-c50b-4c5b-a842-9b1d5c036005 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:38.995112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ec87a-c50b-4c5b-a842-9b1d5c036005 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.011026 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ec87a-c50b-4c5b-a842-9b1d5c036005 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:39.012494 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ec87a-c50b-4c5b-a842-9b1d5c036005 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:39.025780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75082c-c271-448e-be6c-e84f962fa733 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-360ec87a-c50b-4c5b-a842-9b1d5c036005 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:39.027467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:39.028483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:39.030320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:39.030320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:39.030320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:39.030320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:39.030320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:39.030320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:39.030320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:39.030320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:39.030320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:39.030320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:39.030320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 309/619] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 06:17:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:39.034870 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ec87a-c50b-4c5b-a842-9b1d5c036005 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:39.069859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac11575-c949-4253-95ab-42bc96f619c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:39.070619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac11575-c949-4253-95ab-42bc96f619c7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:39.070910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac11575-c949-4253-95ab-42bc96f619c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:39.071189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac11575-c949-4253-95ab-42bc96f619c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.071922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac11575-c949-4253-95ab-42bc96f619c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:39.072230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac11575-c949-4253-95ab-42bc96f619c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:39.073596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac11575-c949-4253-95ab-42bc96f619c7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:39.086130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 311/620] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:39.092937 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac11575-c949-4253-95ab-42bc96f619c7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:39.203111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ec87a-c50b-4c5b-a842-9b1d5c036005 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:39.241448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4fb32-1148-4359-abf3-1060faedbaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:39.242273 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4fb32-1148-4359-abf3-1060faedbaeb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:39.242560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4fb32-1148-4359-abf3-1060faedbaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:39.242892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4fb32-1148-4359-abf3-1060faedbaeb None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.243840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4fb32-1148-4359-abf3-1060faedbaeb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:39.244170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4fb32-1148-4359-abf3-1060faedbaeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:39.246063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4fb32-1148-4359-abf3-1060faedbaeb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:39.265490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 310/621] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 06:17:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:39.266314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:39.272867 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4fb32-1148-4359-abf3-1060faedbaeb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:39.317588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac11575-c949-4253-95ab-42bc96f619c7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:39.335869 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:39.361434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed47596-76fd-480d-b7b9-0a8471de9cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:39.362738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed47596-76fd-480d-b7b9-0a8471de9cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:39.363290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed47596-76fd-480d-b7b9-0a8471de9cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:39.363823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed47596-76fd-480d-b7b9-0a8471de9cb2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.371718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed47596-76fd-480d-b7b9-0a8471de9cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:39.374135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed47596-76fd-480d-b7b9-0a8471de9cb2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6ed47596-76fd-480d-b7b9-0a8471de9cb2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:39.391479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:39.392478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:39.393430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:39.393430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:39.393430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:39.393430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:39.393430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:39.393430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:39.393430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:39.393430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:39.393430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:17:39.393430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:39.400976 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4fb32-1148-4359-abf3-1060faedbaeb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:39.401282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 312/622] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:39.402972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb61d7-1f4f-4f42-a765-8af2fd2b6c7e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:39.403510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb61d7-1f4f-4f42-a765-8af2fd2b6c7e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:39.403983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb61d7-1f4f-4f42-a765-8af2fd2b6c7e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.407266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 313/623] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:39.408349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb61d7-1f4f-4f42-a765-8af2fd2b6c7e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:39.438445 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f0284-b87a-423c-b439-4d20a791d70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:39.439156 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f0284-b87a-423c-b439-4d20a791d70a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:39.439440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f0284-b87a-423c-b439-4d20a791d70a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:39.439716 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f0284-b87a-423c-b439-4d20a791d70a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.501924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:39.511965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f0284-b87a-423c-b439-4d20a791d70a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:39.513481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f0284-b87a-423c-b439-4d20a791d70a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:39.533210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb61d7-1f4f-4f42-a765-8af2fd2b6c7e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:39.574973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e09f6-d519-49c7-b339-c6b8e6852415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:39.575647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e09f6-d519-49c7-b339-c6b8e6852415 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:39.575931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e09f6-d519-49c7-b339-c6b8e6852415 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:39.576209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e09f6-d519-49c7-b339-c6b8e6852415 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.577085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e09f6-d519-49c7-b339-c6b8e6852415 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:39.577382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e09f6-d519-49c7-b339-c6b8e6852415 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:39.578833 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e09f6-d519-49c7-b339-c6b8e6852415 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:39.603133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 314/624] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 06:17:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:39.603133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3a39-06c7-4c90-bf52-d1ab15194357 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:39.603133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3a39-06c7-4c90-bf52-d1ab15194357 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:39.603133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3a39-06c7-4c90-bf52-d1ab15194357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.618440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-935b3a39-06c7-4c90-bf52-d1ab15194357 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:39.619631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-935b3a39-06c7-4c90-bf52-d1ab15194357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:39.625658 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 311/625] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:39.626998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1fd57-be3e-4dcf-ba26-9b91480e2835 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:39.627291 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1fd57-be3e-4dcf-ba26-9b91480e2835 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:39.627567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1fd57-be3e-4dcf-ba26-9b91480e2835 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.629057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 312/626] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:39.630017 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1fd57-be3e-4dcf-ba26-9b91480e2835 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:39.655667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:39.697786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 315/627] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:39.698664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b3a39-06c7-4c90-bf52-d1ab15194357 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:39.740278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:39.752064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1fd57-be3e-4dcf-ba26-9b91480e2835 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:39.788013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d52d40-72cb-4db6-b10d-f0786d038abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:39.789199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d52d40-72cb-4db6-b10d-f0786d038abb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:39.789811 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d52d40-72cb-4db6-b10d-f0786d038abb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:39.790192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d52d40-72cb-4db6-b10d-f0786d038abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.801859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d52d40-72cb-4db6-b10d-f0786d038abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:39.803595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d52d40-72cb-4db6-b10d-f0786d038abb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:39.808592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b3a39-06c7-4c90-bf52-d1ab15194357 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d1d52d40-72cb-4db6-b10d-f0786d038abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:39.823358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:39.824277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:39.825203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:39.825203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:39.825203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:39.825203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:39.825203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 313/628] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:39.826005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc328328-d303-4b75-8aca-191b84c4451e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:39.826005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc328328-d303-4b75-8aca-191b84c4451e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:39.826312 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc328328-d303-4b75-8aca-191b84c4451e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.844462 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dc328328-d303-4b75-8aca-191b84c4451e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:39.845826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dc328328-d303-4b75-8aca-191b84c4451e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:39.850788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7e2d-9889-40d7-bdec-a507236b7098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:39.851588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7e2d-9889-40d7-bdec-a507236b7098 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:39.851873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7e2d-9889-40d7-bdec-a507236b7098 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:39.852148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7e2d-9889-40d7-bdec-a507236b7098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:39.863013 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d7e2d-9889-40d7-bdec-a507236b7098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:39.864450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d7e2d-9889-40d7-bdec-a507236b7098 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2e8d7e2d-9889-40d7-bdec-a507236b7098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:39.882451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:39.883340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:39.884280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:39.884280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:39.884280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:39.884280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:39.884280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 316/629] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:39.885725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7e2d-9889-40d7-bdec-a507236b7098 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:39.903958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:39.917087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 314/630] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:39.917923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc328328-d303-4b75-8aca-191b84c4451e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:39.988701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7e2d-9889-40d7-bdec-a507236b7098 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:39.992945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:40.019175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc328328-d303-4b75-8aca-191b84c4451e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.022218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6b520-95b1-4ea7-bd2b-e0ecd0b73380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.023007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6b520-95b1-4ea7-bd2b-e0ecd0b73380 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.023287 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6b520-95b1-4ea7-bd2b-e0ecd0b73380 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.023563 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6b520-95b1-4ea7-bd2b-e0ecd0b73380 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.034186 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6b520-95b1-4ea7-bd2b-e0ecd0b73380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a9c6b520-95b1-4ea7-bd2b-e0ecd0b73380 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:40.041731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:40.042709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:40.043532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 317/631] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:40.044208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53043842-aeb3-4113-9bec-fe13812735db None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.044886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53043842-aeb3-4113-9bec-fe13812735db None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.044886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53043842-aeb3-4113-9bec-fe13812735db None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.047305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 318/632] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:40.048020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53043842-aeb3-4113-9bec-fe13812735db None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.060316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e806c-6b06-47f5-aa6c-c788aa812090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.061592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e806c-6b06-47f5-aa6c-c788aa812090 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.061874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e806c-6b06-47f5-aa6c-c788aa812090 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.062203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e806c-6b06-47f5-aa6c-c788aa812090 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.062934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e806c-6b06-47f5-aa6c-c788aa812090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:40.063241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e806c-6b06-47f5-aa6c-c788aa812090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:40.064604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e806c-6b06-47f5-aa6c-c788aa812090 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:40.076354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 315/633] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:40.077347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e806c-6b06-47f5-aa6c-c788aa812090 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.129958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:40.154241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53043842-aeb3-4113-9bec-fe13812735db None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.180369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e806c-6b06-47f5-aa6c-c788aa812090 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.189291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead14f-55fb-4017-af9c-bdb69a3fd5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.189964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead14f-55fb-4017-af9c-bdb69a3fd5af None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.190265 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead14f-55fb-4017-af9c-bdb69a3fd5af None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.190545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead14f-55fb-4017-af9c-bdb69a3fd5af None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.191273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ead14f-55fb-4017-af9c-bdb69a3fd5af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:40.191580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ead14f-55fb-4017-af9c-bdb69a3fd5af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:40.192952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ead14f-55fb-4017-af9c-bdb69a3fd5af None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:40.204214 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 319/634] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:40.205006 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead14f-55fb-4017-af9c-bdb69a3fd5af None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.214524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490227a-da8a-4bce-a6c1-dd9198bdd518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.215324 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490227a-da8a-4bce-a6c1-dd9198bdd518 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.215604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490227a-da8a-4bce-a6c1-dd9198bdd518 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.215876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490227a-da8a-4bce-a6c1-dd9198bdd518 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.218829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:40.227948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490227a-da8a-4bce-a6c1-dd9198bdd518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0490227a-da8a-4bce-a6c1-dd9198bdd518 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:40.236786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:40.237742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:40.238629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 316/635] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:40.239617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab798fd-b290-439b-bc9d-99f57d8fe647 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.239617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab798fd-b290-439b-bc9d-99f57d8fe647 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.239739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab798fd-b290-439b-bc9d-99f57d8fe647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.256154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bab798fd-b290-439b-bc9d-99f57d8fe647 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:40.257337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bab798fd-b290-439b-bc9d-99f57d8fe647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:40.310031 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead14f-55fb-4017-af9c-bdb69a3fd5af None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.325214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 317/636] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:40.326229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab798fd-b290-439b-bc9d-99f57d8fe647 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.344177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a365c-39aa-4868-8207-15adf515b2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.344858 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a365c-39aa-4868-8207-15adf515b2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.345149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a365c-39aa-4868-8207-15adf515b2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.345435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a365c-39aa-4868-8207-15adf515b2f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.346234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a365c-39aa-4868-8207-15adf515b2f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:40.346546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a365c-39aa-4868-8207-15adf515b2f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:40.347909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a365c-39aa-4868-8207-15adf515b2f9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:40.350389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:40.358685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 320/637] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:40.359487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a365c-39aa-4868-8207-15adf515b2f9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.434035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab798fd-b290-439b-bc9d-99f57d8fe647 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.457109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a365c-39aa-4868-8207-15adf515b2f9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.457590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:40.469206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41df417-ff5e-4f94-950e-8fce9d7eeae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.470030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41df417-ff5e-4f94-950e-8fce9d7eeae5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.470323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41df417-ff5e-4f94-950e-8fce9d7eeae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.470599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41df417-ff5e-4f94-950e-8fce9d7eeae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.477349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41df417-ff5e-4f94-950e-8fce9d7eeae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:40.478838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41df417-ff5e-4f94-950e-8fce9d7eeae5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:40.491281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860092f1-587c-4646-bcd6-62b02aeb794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.492065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860092f1-587c-4646-bcd6-62b02aeb794c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.492350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860092f1-587c-4646-bcd6-62b02aeb794c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d41df417-ff5e-4f94-950e-8fce9d7eeae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:40.492454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:40.493352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:40.494251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:40.494251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:40.494251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:40.494251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:40.494251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:40.494251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:40.494251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:40.494251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:40.494695 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860092f1-587c-4646-bcd6-62b02aeb794c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.494776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 318/638] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:40.494776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41df417-ff5e-4f94-950e-8fce9d7eeae5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.503638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860092f1-587c-4646-bcd6-62b02aeb794c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:40.505083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860092f1-587c-4646-bcd6-62b02aeb794c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-860092f1-587c-4646-bcd6-62b02aeb794c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:40.519803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:40.520764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:40.521715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:40.521715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:40.521715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:40.521715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:40.521715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:40.521715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:40.521715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:40.521715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:40.521715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 321/639] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:40.522224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860092f1-587c-4646-bcd6-62b02aeb794c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.581723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:40.597816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41df417-ff5e-4f94-950e-8fce9d7eeae5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.624199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860092f1-587c-4646-bcd6-62b02aeb794c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.632346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f4db8-f907-4101-9811-511d9171c6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.633014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f4db8-f907-4101-9811-511d9171c6fe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.633301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f4db8-f907-4101-9811-511d9171c6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.633586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f4db8-f907-4101-9811-511d9171c6fe None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.634370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f4db8-f907-4101-9811-511d9171c6fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:40.634684 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f4db8-f907-4101-9811-511d9171c6fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:40.636055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f4db8-f907-4101-9811-511d9171c6fe None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:40.643866 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 319/640] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:40.644655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f4db8-f907-4101-9811-511d9171c6fe None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.657150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f8f11-d32f-404c-9fba-1a508ef6dc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.657961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f8f11-d32f-404c-9fba-1a508ef6dc19 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.658247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f8f11-d32f-404c-9fba-1a508ef6dc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.658524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f8f11-d32f-404c-9fba-1a508ef6dc19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.668269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f8f11-d32f-404c-9fba-1a508ef6dc19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d76f8f11-d32f-404c-9fba-1a508ef6dc19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:40.676546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:40.677742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 322/641] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:40.678641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f8f11-d32f-404c-9fba-1a508ef6dc19 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.695337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:40.749306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f4db8-f907-4101-9811-511d9171c6fe None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.777865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f8f11-d32f-404c-9fba-1a508ef6dc19 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.783370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b58bff-704c-4cca-bb77-1e8f43cc3038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.784169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58bff-704c-4cca-bb77-1e8f43cc3038 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.784454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58bff-704c-4cca-bb77-1e8f43cc3038 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.784733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58bff-704c-4cca-bb77-1e8f43cc3038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.795526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b58bff-704c-4cca-bb77-1e8f43cc3038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:40.795845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:40.797401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b58bff-704c-4cca-bb77-1e8f43cc3038 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:40.813812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5259d-bbf5-48a4-87c3-d7c1a4ef215e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.814511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5259d-bbf5-48a4-87c3-d7c1a4ef215e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.814797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5259d-bbf5-48a4-87c3-d7c1a4ef215e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.815076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5259d-bbf5-48a4-87c3-d7c1a4ef215e None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.815870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5259d-bbf5-48a4-87c3-d7c1a4ef215e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:40.816256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5259d-bbf5-48a4-87c3-d7c1a4ef215e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b0b58bff-704c-4cca-bb77-1e8f43cc3038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:40.816372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:40.817371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:40.818398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:40.818398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:40.818398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:40.818398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:40.818625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5259d-bbf5-48a4-87c3-d7c1a4ef215e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:40.818703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 320/642] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:40.818703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b58bff-704c-4cca-bb77-1e8f43cc3038 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.835022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 323/643] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:40.835768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5259d-bbf5-48a4-87c3-d7c1a4ef215e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.922010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:40.926500 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b58bff-704c-4cca-bb77-1e8f43cc3038 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.933511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5259d-bbf5-48a4-87c3-d7c1a4ef215e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:40.960490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d92e71-150a-48e1-974f-9e459c557690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.961145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d92e71-150a-48e1-974f-9e459c557690 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.961427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d92e71-150a-48e1-974f-9e459c557690 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.961703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d92e71-150a-48e1-974f-9e459c557690 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.962537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d92e71-150a-48e1-974f-9e459c557690 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:40.962847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d92e71-150a-48e1-974f-9e459c557690 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:40.964275 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d92e71-150a-48e1-974f-9e459c557690 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:40.967781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd2359-4082-43b8-8db4-7c33db8911a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:40.968562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2359-4082-43b8-8db4-7c33db8911a9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:40.968844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2359-4082-43b8-8db4-7c33db8911a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:40.969119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2359-4082-43b8-8db4-7c33db8911a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:40.976115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 321/644] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:40.976914 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d92e71-150a-48e1-974f-9e459c557690 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:40.983444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd2359-4082-43b8-8db4-7c33db8911a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:40.984906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd2359-4082-43b8-8db4-7c33db8911a9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-31bd2359-4082-43b8-8db4-7c33db8911a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:41.002914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:41.003824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:41.004721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:41.004721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:41.004721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:41.004721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:41.004721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 324/645] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:41.004991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd2359-4082-43b8-8db4-7c33db8911a9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:41.034633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:41.082563 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d92e71-150a-48e1-974f-9e459c557690 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:41.108502 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd2359-4082-43b8-8db4-7c33db8911a9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:41.115741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2edba-34f7-4633-8850-30b34ba0c887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:41.116409 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2edba-34f7-4633-8850-30b34ba0c887 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:41.116694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2edba-34f7-4633-8850-30b34ba0c887 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:41.116973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2edba-34f7-4633-8850-30b34ba0c887 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:41.117724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2edba-34f7-4633-8850-30b34ba0c887 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:41.118070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2edba-34f7-4633-8850-30b34ba0c887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:41.119439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2edba-34f7-4633-8850-30b34ba0c887 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:41.127918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 322/646] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:41.128682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2edba-34f7-4633-8850-30b34ba0c887 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:41.147304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0802a9b-2f30-43ce-8563-6682cf894cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:41.147968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0802a9b-2f30-43ce-8563-6682cf894cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:41.148255 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0802a9b-2f30-43ce-8563-6682cf894cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:41.148535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0802a9b-2f30-43ce-8563-6682cf894cb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:41.149512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0802a9b-2f30-43ce-8563-6682cf894cb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:41.149820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0802a9b-2f30-43ce-8563-6682cf894cb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:41.150874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:41.151231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0802a9b-2f30-43ce-8563-6682cf894cb6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:41.162529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 325/647] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:41.163325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0802a9b-2f30-43ce-8563-6682cf894cb6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:41.230687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2edba-34f7-4633-8850-30b34ba0c887 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:41.253198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:41.263582 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0802a9b-2f30-43ce-8563-6682cf894cb6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:41.264841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77685477-a205-4ee3-9ac7-87b17513fce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:41.265819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77685477-a205-4ee3-9ac7-87b17513fce5 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:41.266193 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77685477-a205-4ee3-9ac7-87b17513fce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:41.266540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77685477-a205-4ee3-9ac7-87b17513fce5 None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/94c8eb61f2ca48288cba81d280a30747/roles/7541347b464141a78014fd57dccc765b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:41.281083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77685477-a205-4ee3-9ac7-87b17513fce5 None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=94c8eb61f2ca48288cba81d280a30747, role_id=7541347b464141a78014fd57dccc765b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:41.282569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77685477-a205-4ee3-9ac7-87b17513fce5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:41.294394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 323/648] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:17:41 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/94c8eb61f2ca48288cba81d280a30747/roles/7541347b464141a78014fd57dccc765b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:17:41.295796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77685477-a205-4ee3-9ac7-87b17513fce5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:41.298389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cc932-334b-4a81-a2f0-dfcccc5750df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:41.299061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cc932-334b-4a81-a2f0-dfcccc5750df None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:41.299346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cc932-334b-4a81-a2f0-dfcccc5750df None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:41.300443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cc932-334b-4a81-a2f0-dfcccc5750df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:41.300561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cc932-334b-4a81-a2f0-dfcccc5750df None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=058f099e6f7545a1b2734dbe5afd2027) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:41.300784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cc932-334b-4a81-a2f0-dfcccc5750df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:41.302280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cc932-334b-4a81-a2f0-dfcccc5750df None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:41.314153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 326/649] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:41 2022] GET /identity/v3/role_assignments?user.id=058f099e6f7545a1b2734dbe5afd2027&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:41.315176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cc932-334b-4a81-a2f0-dfcccc5750df None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:41.389922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:41.433257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77685477-a205-4ee3-9ac7-87b17513fce5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:41.466693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cc932-334b-4a81-a2f0-dfcccc5750df None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:41.476492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c87d3b-1b5d-4b37-acbf-0d458a229d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:41.476900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c87d3b-1b5d-4b37-acbf-0d458a229d06 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:41.477228 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c87d3b-1b5d-4b37-acbf-0d458a229d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:41.477509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c87d3b-1b5d-4b37-acbf-0d458a229d06 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:41.479404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c87d3b-1b5d-4b37-acbf-0d458a229d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:41.479539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c87d3b-1b5d-4b37-acbf-0d458a229d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:41.481134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c87d3b-1b5d-4b37-acbf-0d458a229d06 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:41.495828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 324/650] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:41.496699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c87d3b-1b5d-4b37-acbf-0d458a229d06 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:41.514435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f56791-71a9-4cb1-bc2a-e277b7db92bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:41.515126 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f56791-71a9-4cb1-bc2a-e277b7db92bc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:41.515411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f56791-71a9-4cb1-bc2a-e277b7db92bc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:41.515743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f56791-71a9-4cb1-bc2a-e277b7db92bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:41.530853 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f56791-71a9-4cb1-bc2a-e277b7db92bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:41.532867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f56791-71a9-4cb1-bc2a-e277b7db92bc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-39f56791-71a9-4cb1-bc2a-e277b7db92bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:41.553146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:41.554131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:41.555037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:41.555037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:41.555037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:41.555037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:41.555037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 327/651] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:41.555814 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f56791-71a9-4cb1-bc2a-e277b7db92bc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:41.567668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:41.642958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c87d3b-1b5d-4b37-acbf-0d458a229d06 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:41.673113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:41.679671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcaadd-8ee6-45e6-935f-0e4bc4d50280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:41.680476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcaadd-8ee6-45e6-935f-0e4bc4d50280 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:41.680935 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcaadd-8ee6-45e6-935f-0e4bc4d50280 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:41.681565 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f56791-71a9-4cb1-bc2a-e277b7db92bc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:41.681774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcaadd-8ee6-45e6-935f-0e4bc4d50280 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:41.690781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcaadd-8ee6-45e6-935f-0e4bc4d50280 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:41.692417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcaadd-8ee6-45e6-935f-0e4bc4d50280 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-aebcaadd-8ee6-45e6-935f-0e4bc4d50280 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:41.708121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:41.709314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:41.710510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:41.710510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:41.710510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:41.710510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:41.710510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:41.710510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:41.710510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:41.710510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:41.710510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 06:17:41.710510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:41.716111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 325/652] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:41.717683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcaadd-8ee6-45e6-935f-0e4bc4d50280 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:41.723719 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0c98a-2bbc-4fb2-87a2-65c827408e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:41.724730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0c98a-2bbc-4fb2-87a2-65c827408e70 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:41.725193 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0c98a-2bbc-4fb2-87a2-65c827408e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:41.725621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0c98a-2bbc-4fb2-87a2-65c827408e70 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:41.734664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0c98a-2bbc-4fb2-87a2-65c827408e70 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:41.736381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0c98a-2bbc-4fb2-87a2-65c827408e70 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-02a0c98a-2bbc-4fb2-87a2-65c827408e70 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:41.754620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:41.755616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:41.756627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:41.756627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:41.756627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:41.756627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:41.756627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:41.756627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:41.756627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:41.756627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:41.756627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 06:17:41.756627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:41.759288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 328/653] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:41.760412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0c98a-2bbc-4fb2-87a2-65c827408e70 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:41.828067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcaadd-8ee6-45e6-935f-0e4bc4d50280 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:41.839703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:41.872496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058debbb-ad55-4c09-b84e-482d75221283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:41.873738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058debbb-ad55-4c09-b84e-482d75221283 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:41.874289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058debbb-ad55-4c09-b84e-482d75221283 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:41.874579 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058debbb-ad55-4c09-b84e-482d75221283 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:41.875321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058debbb-ad55-4c09-b84e-482d75221283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:41.875634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058debbb-ad55-4c09-b84e-482d75221283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:41.877513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058debbb-ad55-4c09-b84e-482d75221283 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:41.886671 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0c98a-2bbc-4fb2-87a2-65c827408e70 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:41.898184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 326/654] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:41.898184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058debbb-ad55-4c09-b84e-482d75221283 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:41.939874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750637f7-4c85-4971-99fb-cc7ef0e063df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:41.940967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750637f7-4c85-4971-99fb-cc7ef0e063df None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:41.941462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750637f7-4c85-4971-99fb-cc7ef0e063df None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:41.941978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750637f7-4c85-4971-99fb-cc7ef0e063df None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:41.943579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750637f7-4c85-4971-99fb-cc7ef0e063df None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:41.944137 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750637f7-4c85-4971-99fb-cc7ef0e063df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:41.946680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750637f7-4c85-4971-99fb-cc7ef0e063df None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:41.954396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:41.974000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 329/655] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 06:17:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:41.975449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750637f7-4c85-4971-99fb-cc7ef0e063df None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:42.017825 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058debbb-ad55-4c09-b84e-482d75221283 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:42.067304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c6f60-aa5b-49c3-89e1-412c8768b4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:42.068496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c6f60-aa5b-49c3-89e1-412c8768b4fc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:42.069382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c6f60-aa5b-49c3-89e1-412c8768b4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:42.069891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c6f60-aa5b-49c3-89e1-412c8768b4fc None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:42.071003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c6f60-aa5b-49c3-89e1-412c8768b4fc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:42.071523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c6f60-aa5b-49c3-89e1-412c8768b4fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:42.074040 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c6f60-aa5b-49c3-89e1-412c8768b4fc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:42.085814 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750637f7-4c85-4971-99fb-cc7ef0e063df None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:42.100299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 327/656] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Jul 21 06:17:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:42.101683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c6f60-aa5b-49c3-89e1-412c8768b4fc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:42.120476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:42.141582 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994660a-7840-4372-8c0a-68fedf38b501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:42.142431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994660a-7840-4372-8c0a-68fedf38b501 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:42.142717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994660a-7840-4372-8c0a-68fedf38b501 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:42.142996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994660a-7840-4372-8c0a-68fedf38b501 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:42.151565 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994660a-7840-4372-8c0a-68fedf38b501 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:42.152411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994660a-7840-4372-8c0a-68fedf38b501 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8994660a-7840-4372-8c0a-68fedf38b501 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:42.168168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:42.169128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:42.170744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:42.170744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:42.170744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:42.170744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:42.170744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:42.170744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:42.170744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:42.170744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:42.170744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 06:17:42.170744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:42.174187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 330/657] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:42.175029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994660a-7840-4372-8c0a-68fedf38b501 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:42.243842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c6f60-aa5b-49c3-89e1-412c8768b4fc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:42.257264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:42.283774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d62c6-151d-45b1-868d-1e8ece0fb38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:42.284532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d62c6-151d-45b1-868d-1e8ece0fb38c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:42.284946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d62c6-151d-45b1-868d-1e8ece0fb38c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:42.285325 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d62c6-151d-45b1-868d-1e8ece0fb38c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:42.294140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994660a-7840-4372-8c0a-68fedf38b501 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:42.339233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba920f73-2a71-449d-a1ad-364dcb30d982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:42.340501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920f73-2a71-449d-a1ad-364dcb30d982 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:42.340955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920f73-2a71-449d-a1ad-364dcb30d982 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:42.341409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920f73-2a71-449d-a1ad-364dcb30d982 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:42.358544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d62c6-151d-45b1-868d-1e8ece0fb38c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:42.360183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d62c6-151d-45b1-868d-1e8ece0fb38c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:42.409680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:42.431451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba920f73-2a71-449d-a1ad-364dcb30d982 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:42.434115 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba920f73-2a71-449d-a1ad-364dcb30d982 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:42.471492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 328/658] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:42.472585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d62c6-151d-45b1-868d-1e8ece0fb38c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:42.483989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:42.564253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 331/659] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:42.565433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba920f73-2a71-449d-a1ad-364dcb30d982 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:42.591097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d62c6-151d-45b1-868d-1e8ece0fb38c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:42.630164 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413424f-a794-4f98-8eb1-1b56c606d9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:42.631009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413424f-a794-4f98-8eb1-1b56c606d9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:42.631649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413424f-a794-4f98-8eb1-1b56c606d9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:42.632031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413424f-a794-4f98-8eb1-1b56c606d9e6 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:42.633089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413424f-a794-4f98-8eb1-1b56c606d9e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:42.633503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413424f-a794-4f98-8eb1-1b56c606d9e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:42.635073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413424f-a794-4f98-8eb1-1b56c606d9e6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:42.659584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 329/660] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 06:17:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:42.660638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:42.661694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413424f-a794-4f98-8eb1-1b56c606d9e6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:42.673400 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba920f73-2a71-449d-a1ad-364dcb30d982 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:42.710109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d751d9c-0d57-477d-9be1-04591238fde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:42.711124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d751d9c-0d57-477d-9be1-04591238fde6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:42.711559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d751d9c-0d57-477d-9be1-04591238fde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:42.711983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d751d9c-0d57-477d-9be1-04591238fde6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:42.722608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d751d9c-0d57-477d-9be1-04591238fde6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:42.724225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d751d9c-0d57-477d-9be1-04591238fde6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:42.734991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2d751d9c-0d57-477d-9be1-04591238fde6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:42.743024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:42.743901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:42.744795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:42.744795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:42.744795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:42.744795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:42.747168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 332/661] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:42.748820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d751d9c-0d57-477d-9be1-04591238fde6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:42.773606 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413424f-a794-4f98-8eb1-1b56c606d9e6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:42.812981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4cdd8-b198-4724-93c0-24d0d9c9b013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:42.813787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4cdd8-b198-4724-93c0-24d0d9c9b013 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:42.814116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4cdd8-b198-4724-93c0-24d0d9c9b013 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:42.814394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4cdd8-b198-4724-93c0-24d0d9c9b013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:42.825632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4cdd8-b198-4724-93c0-24d0d9c9b013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:42.827232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4cdd8-b198-4724-93c0-24d0d9c9b013 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-07f4cdd8-b198-4724-93c0-24d0d9c9b013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:42.846694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:42.847624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:42.848604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:42.848604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:42.848604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:42.848604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:42.852466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 330/662] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:42.853557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4cdd8-b198-4724-93c0-24d0d9c9b013 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:42.859422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d751d9c-0d57-477d-9be1-04591238fde6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:42.895341 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37723f86-bd82-49ff-b738-628a17ea2f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:42.896366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37723f86-bd82-49ff-b738-628a17ea2f97 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:42.896880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37723f86-bd82-49ff-b738-628a17ea2f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:42.897372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37723f86-bd82-49ff-b738-628a17ea2f97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:42.901389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:42.967986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4cdd8-b198-4724-93c0-24d0d9c9b013 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:42.968801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37723f86-bd82-49ff-b738-628a17ea2f97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:42.970393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37723f86-bd82-49ff-b738-628a17ea2f97 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:42.988389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:43.009172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c02b2-c1b4-4930-914c-11cb391be5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:43.010146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c02b2-c1b4-4930-914c-11cb391be5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.010592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c02b2-c1b4-4930-914c-11cb391be5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.011053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c02b2-c1b4-4930-914c-11cb391be5a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.011930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c02b2-c1b4-4930-914c-11cb391be5a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:43.012484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c02b2-c1b4-4930-914c-11cb391be5a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:43.014235 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c02b2-c1b4-4930-914c-11cb391be5a2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:43.027983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 331/663] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:43.029335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c02b2-c1b4-4930-914c-11cb391be5a2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:43.076339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 333/664] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:43.077240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37723f86-bd82-49ff-b738-628a17ea2f97 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:43.113459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:43.151773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c02b2-c1b4-4930-914c-11cb391be5a2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:43.182236 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37723f86-bd82-49ff-b738-628a17ea2f97 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:43.190249 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba1fcd-7c47-4ce6-8848-0475666c4862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:43.191223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba1fcd-7c47-4ce6-8848-0475666c4862 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.191655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba1fcd-7c47-4ce6-8848-0475666c4862 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.192079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba1fcd-7c47-4ce6-8848-0475666c4862 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.193063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba1fcd-7c47-4ce6-8848-0475666c4862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:43.193531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba1fcd-7c47-4ce6-8848-0475666c4862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:43.195199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba1fcd-7c47-4ce6-8848-0475666c4862 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:43.213331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 332/665] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:43.214639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba1fcd-7c47-4ce6-8848-0475666c4862 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:43.218716 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d60884-0092-4708-b212-3d2d9c33f3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:43.220127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d60884-0092-4708-b212-3d2d9c33f3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.220127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d60884-0092-4708-b212-3d2d9c33f3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.220127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d60884-0092-4708-b212-3d2d9c33f3c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.227525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d60884-0092-4708-b212-3d2d9c33f3c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:43.228674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d60884-0092-4708-b212-3d2d9c33f3c4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-40d60884-0092-4708-b212-3d2d9c33f3c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:43.243780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:43.244915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:43.246049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:43.246049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:43.246049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:43.246049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:43.246049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:43.246049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:43.246049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:43.246049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:43.246049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 334/666] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:43.246049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d60884-0092-4708-b212-3d2d9c33f3c4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:43.263868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:43.334251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba1fcd-7c47-4ce6-8848-0475666c4862 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:43.347244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:43.353074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d60884-0092-4708-b212-3d2d9c33f3c4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:43.375997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660806d4-44c3-4e59-bdb7-6af29e58e034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:43.377134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660806d4-44c3-4e59-bdb7-6af29e58e034 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.377565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660806d4-44c3-4e59-bdb7-6af29e58e034 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.378041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660806d4-44c3-4e59-bdb7-6af29e58e034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.390324 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660806d4-44c3-4e59-bdb7-6af29e58e034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:43.391775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a64e7-2609-40e4-b408-474e9c70131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:43.392512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660806d4-44c3-4e59-bdb7-6af29e58e034 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:43.392647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a64e7-2609-40e4-b408-474e9c70131f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.392858 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a64e7-2609-40e4-b408-474e9c70131f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.393136 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a64e7-2609-40e4-b408-474e9c70131f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.400875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a64e7-2609-40e4-b408-474e9c70131f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:43.402410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a64e7-2609-40e4-b408-474e9c70131f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-660806d4-44c3-4e59-bdb7-6af29e58e034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:43.413133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:43.414242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:43.415301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:43.415301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:43.415301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:43.415301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0d1a64e7-2609-40e4-b408-474e9c70131f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:43.416638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:43.417472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:43.418414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:43.418414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:43.418414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:43.418414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:43.418414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:43.418414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:43.418414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:43.418414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:43.423771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 335/668] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:43.423771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a64e7-2609-40e4-b408-474e9c70131f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:43.424051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 333/668] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:43.425050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660806d4-44c3-4e59-bdb7-6af29e58e034 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:43.522090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:43.531308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a64e7-2609-40e4-b408-474e9c70131f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:43.533398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660806d4-44c3-4e59-bdb7-6af29e58e034 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:43.568357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beea8ae-134a-4487-9b53-c36bef7aee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:43.569018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beea8ae-134a-4487-9b53-c36bef7aee22 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.569301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beea8ae-134a-4487-9b53-c36bef7aee22 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.569574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beea8ae-134a-4487-9b53-c36bef7aee22 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.570392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beea8ae-134a-4487-9b53-c36bef7aee22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:43.570701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beea8ae-134a-4487-9b53-c36bef7aee22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:43.572133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beea8ae-134a-4487-9b53-c36bef7aee22 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:43.573014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31eb8d-0aa4-450f-88be-8a5dfd631f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:43.573686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31eb8d-0aa4-450f-88be-8a5dfd631f4e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.573996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31eb8d-0aa4-450f-88be-8a5dfd631f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.574284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31eb8d-0aa4-450f-88be-8a5dfd631f4e None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.575083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31eb8d-0aa4-450f-88be-8a5dfd631f4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:43.575395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31eb8d-0aa4-450f-88be-8a5dfd631f4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:43.576947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31eb8d-0aa4-450f-88be-8a5dfd631f4e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:43.584006 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 336/669] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:43.584798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beea8ae-134a-4487-9b53-c36bef7aee22 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:43.588812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 334/670] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:43.590105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9774549-bd81-4879-b20e-d43dd0f239be None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.590393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9774549-bd81-4879-b20e-d43dd0f239be None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.590668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9774549-bd81-4879-b20e-d43dd0f239be None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.594424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 335/671] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:43.594424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3807ff8-68b5-4ef7-94c2-44bfdff38aa4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.594424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3807ff8-68b5-4ef7-94c2-44bfdff38aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.594424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3807ff8-68b5-4ef7-94c2-44bfdff38aa4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.595477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 336/672] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:43.599078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3807ff8-68b5-4ef7-94c2-44bfdff38aa4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:43.603810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:43.686226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beea8ae-134a-4487-9b53-c36bef7aee22 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:43.701514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3807ff8-68b5-4ef7-94c2-44bfdff38aa4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:43.719179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8294b5-323a-4492-a421-e09a3e8dc466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:43.719844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8294b5-323a-4492-a421-e09a3e8dc466 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.720134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8294b5-323a-4492-a421-e09a3e8dc466 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.720417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8294b5-323a-4492-a421-e09a3e8dc466 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.721156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8294b5-323a-4492-a421-e09a3e8dc466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:43.721468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8294b5-323a-4492-a421-e09a3e8dc466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:43.722872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8294b5-323a-4492-a421-e09a3e8dc466 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:43.734490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cff5cb-de28-4d6f-9713-8aa35404aa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:43.734647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 337/673] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:43.735146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cff5cb-de28-4d6f-9713-8aa35404aa70 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.735361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8294b5-323a-4492-a421-e09a3e8dc466 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:43.735471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cff5cb-de28-4d6f-9713-8aa35404aa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.735704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cff5cb-de28-4d6f-9713-8aa35404aa70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.736526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cff5cb-de28-4d6f-9713-8aa35404aa70 None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=710c422b8024403da666ae5054ba1289) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:43.736839 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cff5cb-de28-4d6f-9713-8aa35404aa70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:43.738287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cff5cb-de28-4d6f-9713-8aa35404aa70 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:43.757982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 337/674] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:43 2022] GET /identity/v3/role_assignments?user.id=710c422b8024403da666ae5054ba1289&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:43.757982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59899e2f-86a4-4fa9-b9bf-935f11dfb3b9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.757982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59899e2f-86a4-4fa9-b9bf-935f11dfb3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.757982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59899e2f-86a4-4fa9-b9bf-935f11dfb3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.763474 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-59899e2f-86a4-4fa9-b9bf-935f11dfb3b9 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:43.764725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-59899e2f-86a4-4fa9-b9bf-935f11dfb3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:43.775869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:43.823451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:43.847393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 338/675] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:43.848962 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d7d0f-dee8-4c9b-b7f1-b303997fa498 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.849271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d7d0f-dee8-4c9b-b7f1-b303997fa498 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.849552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d7d0f-dee8-4c9b-b7f1-b303997fa498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.861100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8294b5-323a-4492-a421-e09a3e8dc466 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:43.875032 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8c0d7d0f-dee8-4c9b-b7f1-b303997fa498 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:43.876269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8c0d7d0f-dee8-4c9b-b7f1-b303997fa498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:43.915838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e456428-780b-4c35-a7c3-c52d6d26a1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:43.916736 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e456428-780b-4c35-a7c3-c52d6d26a1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:43.917086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e456428-780b-4c35-a7c3-c52d6d26a1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:43.917457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e456428-780b-4c35-a7c3-c52d6d26a1b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:43.918444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e456428-780b-4c35-a7c3-c52d6d26a1b2 None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=1a9c3f34eaeb4b538528ff9050252dc7) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:43.918863 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e456428-780b-4c35-a7c3-c52d6d26a1b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:43.921183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e456428-780b-4c35-a7c3-c52d6d26a1b2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:43.928119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 338/676] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:43 2022] GET /identity/v3/role_assignments?user.id=1a9c3f34eaeb4b538528ff9050252dc7&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:43.929528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e456428-780b-4c35-a7c3-c52d6d26a1b2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:43.965926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 339/677] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:43.965926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d7d0f-dee8-4c9b-b7f1-b303997fa498 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:44.073884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e456428-780b-4c35-a7c3-c52d6d26a1b2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:44.080045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:44.118170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d7d0f-dee8-4c9b-b7f1-b303997fa498 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:44.120169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8a38-d5e7-4da9-9850-ba1cae39c03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:44.121899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8a38-d5e7-4da9-9850-ba1cae39c03a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:44.122236 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8a38-d5e7-4da9-9850-ba1cae39c03a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:44.122521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8a38-d5e7-4da9-9850-ba1cae39c03a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:44.131784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b8a38-d5e7-4da9-9850-ba1cae39c03a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:44.133536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b8a38-d5e7-4da9-9850-ba1cae39c03a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:44.136892 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-769b8a38-d5e7-4da9-9850-ba1cae39c03a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:44.154976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:44.155905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:44.158750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:44.158750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:44.158750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:44.158750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:44.158750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:44.158750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:44.158750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:44.158750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:44.158750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 339/678] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:44.158750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8a38-d5e7-4da9-9850-ba1cae39c03a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:44.172725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0acbc-02da-4309-ad9a-78f7d763e93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:44.172725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0acbc-02da-4309-ad9a-78f7d763e93c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:44.172986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0acbc-02da-4309-ad9a-78f7d763e93c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:44.173205 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0acbc-02da-4309-ad9a-78f7d763e93c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:44.186934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0acbc-02da-4309-ad9a-78f7d763e93c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6da0acbc-02da-4309-ad9a-78f7d763e93c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:44.195560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:44.196508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:44.197487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 340/679] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:44.198227 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0acbc-02da-4309-ad9a-78f7d763e93c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:44.288455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8a38-d5e7-4da9-9850-ba1cae39c03a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:44.312872 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0acbc-02da-4309-ad9a-78f7d763e93c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:44.331352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d871f-a6c2-4e23-8eef-0ad7274bec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:44.332159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d871f-a6c2-4e23-8eef-0ad7274bec00 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:44.332510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d871f-a6c2-4e23-8eef-0ad7274bec00 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:44.332857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d871f-a6c2-4e23-8eef-0ad7274bec00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:44.344933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d871f-a6c2-4e23-8eef-0ad7274bec00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f78d871f-a6c2-4e23-8eef-0ad7274bec00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:44.352601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:44.353534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:44.354447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:44.358182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 340/680] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:44.359019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d871f-a6c2-4e23-8eef-0ad7274bec00 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:44.364872 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd8060-f46a-4725-b02f-721bb458b8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:44.365816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8060-f46a-4725-b02f-721bb458b8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:44.366333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8060-f46a-4725-b02f-721bb458b8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:44.366822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8060-f46a-4725-b02f-721bb458b8c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:44.367823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd8060-f46a-4725-b02f-721bb458b8c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:44.368358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd8060-f46a-4725-b02f-721bb458b8c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:44.370485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd8060-f46a-4725-b02f-721bb458b8c9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:44.374320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:44.387478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 341/681] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:44.389532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd8060-f46a-4725-b02f-721bb458b8c9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:44.424253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:44.467733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d871f-a6c2-4e23-8eef-0ad7274bec00 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:44.512144 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfe54d-6282-4966-9eb5-bfc55b91a6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:44.512966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfe54d-6282-4966-9eb5-bfc55b91a6ed None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:44.513366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfe54d-6282-4966-9eb5-bfc55b91a6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:44.513743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfe54d-6282-4966-9eb5-bfc55b91a6ed None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:44.514713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfe54d-6282-4966-9eb5-bfc55b91a6ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:44.515159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfe54d-6282-4966-9eb5-bfc55b91a6ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:44.516631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfe54d-6282-4966-9eb5-bfc55b91a6ed None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:44.521184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd8060-f46a-4725-b02f-721bb458b8c9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:44.530534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 341/682] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:44.531536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfe54d-6282-4966-9eb5-bfc55b91a6ed None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:44.567784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b2eb7-a8dd-4a17-b121-1ec8c25ae64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:44.568464 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b2eb7-a8dd-4a17-b121-1ec8c25ae64c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:44.568750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b2eb7-a8dd-4a17-b121-1ec8c25ae64c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:44.569031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b2eb7-a8dd-4a17-b121-1ec8c25ae64c None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:44.570071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b2eb7-a8dd-4a17-b121-1ec8c25ae64c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:44.571282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b2eb7-a8dd-4a17-b121-1ec8c25ae64c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:44.572668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b2eb7-a8dd-4a17-b121-1ec8c25ae64c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:44.585272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 342/683] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:44.586175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b2eb7-a8dd-4a17-b121-1ec8c25ae64c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:44.645554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfe54d-6282-4966-9eb5-bfc55b91a6ed None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:44.663963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:44.671829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:44.686714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d890e3-bb2c-4dfe-8fa4-c0d68377895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:44.687527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d890e3-bb2c-4dfe-8fa4-c0d68377895e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:44.687950 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d890e3-bb2c-4dfe-8fa4-c0d68377895e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:44.688328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d890e3-bb2c-4dfe-8fa4-c0d68377895e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:44.689265 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d890e3-bb2c-4dfe-8fa4-c0d68377895e None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=fa2c8be3105a415ead65579dc18ba40d) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:44.689682 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d890e3-bb2c-4dfe-8fa4-c0d68377895e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:44.693250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d890e3-bb2c-4dfe-8fa4-c0d68377895e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:44.703494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 342/684] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:44 2022] GET /identity/v3/role_assignments?user.id=fa2c8be3105a415ead65579dc18ba40d&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:44.704471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d890e3-bb2c-4dfe-8fa4-c0d68377895e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:44.721748 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b2eb7-a8dd-4a17-b121-1ec8c25ae64c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:44.796580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d7aa9-e597-4d58-8bc5-a1b22523ddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:44.797729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d7aa9-e597-4d58-8bc5-a1b22523ddd2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:44.798281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d7aa9-e597-4d58-8bc5-a1b22523ddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:44.798712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d7aa9-e597-4d58-8bc5-a1b22523ddd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:44.823165 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d7aa9-e597-4d58-8bc5-a1b22523ddd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:44.826526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d7aa9-e597-4d58-8bc5-a1b22523ddd2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8e5d7aa9-e597-4d58-8bc5-a1b22523ddd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:44.860782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:44.861927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:44.863087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:44.863087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:44.863087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:44.863087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:44.870584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 343/685] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:44.876720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d7aa9-e597-4d58-8bc5-a1b22523ddd2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:44.918539 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d890e3-bb2c-4dfe-8fa4-c0d68377895e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:44.963461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799abf0-25b6-40c0-8e48-1f320aff0e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:44.964408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799abf0-25b6-40c0-8e48-1f320aff0e4d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:44.964815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799abf0-25b6-40c0-8e48-1f320aff0e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:44.965191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799abf0-25b6-40c0-8e48-1f320aff0e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:44.978046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799abf0-25b6-40c0-8e48-1f320aff0e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:44.979650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799abf0-25b6-40c0-8e48-1f320aff0e4d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8799abf0-25b6-40c0-8e48-1f320aff0e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:44.998279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:44.999377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:45.000430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:45.000430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:45.000430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:45.000430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:45.002165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 343/686] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:45.007396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799abf0-25b6-40c0-8e48-1f320aff0e4d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:45.026133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d7aa9-e597-4d58-8bc5-a1b22523ddd2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:45.039063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:45.062500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:45.074748 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff5d32-af70-4ac4-a73b-b34a6020d3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:45.076375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff5d32-af70-4ac4-a73b-b34a6020d3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:45.076567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff5d32-af70-4ac4-a73b-b34a6020d3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:45.076949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff5d32-af70-4ac4-a73b-b34a6020d3c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:45.077810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff5d32-af70-4ac4-a73b-b34a6020d3c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:45.078252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff5d32-af70-4ac4-a73b-b34a6020d3c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:45.079733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff5d32-af70-4ac4-a73b-b34a6020d3c7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:45.095274 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 344/687] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:45.100733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff5d32-af70-4ac4-a73b-b34a6020d3c7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:45.145964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799abf0-25b6-40c0-8e48-1f320aff0e4d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:45.195707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a24059-b42f-4f48-88cd-65975553b307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:45.196810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a24059-b42f-4f48-88cd-65975553b307 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:45.198041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a24059-b42f-4f48-88cd-65975553b307 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:45.198476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a24059-b42f-4f48-88cd-65975553b307 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:45.199377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a24059-b42f-4f48-88cd-65975553b307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:45.199832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a24059-b42f-4f48-88cd-65975553b307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:45.201403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a24059-b42f-4f48-88cd-65975553b307 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:45.216282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 344/688] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:45.222684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a24059-b42f-4f48-88cd-65975553b307 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:45.233959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff5d32-af70-4ac4-a73b-b34a6020d3c7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:45.277895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055dbda-a96d-4ad2-9b30-404224b14d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:45.279359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055dbda-a96d-4ad2-9b30-404224b14d8b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:45.279786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055dbda-a96d-4ad2-9b30-404224b14d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:45.280208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055dbda-a96d-4ad2-9b30-404224b14d8b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:45.289856 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055dbda-a96d-4ad2-9b30-404224b14d8b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:45.291540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055dbda-a96d-4ad2-9b30-404224b14d8b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8055dbda-a96d-4ad2-9b30-404224b14d8b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:45.307381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:45.308581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:45.309741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:45.309741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:45.309741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:45.309741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:45.309741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:45.309741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:45.309741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:45.309741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:45.309741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:17:45.309741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:45.312411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 345/689] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:45.322662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055dbda-a96d-4ad2-9b30-404224b14d8b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:45.338382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:45.344165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a24059-b42f-4f48-88cd-65975553b307 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:45.384275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a536f-cc42-4fbf-b858-72cba256f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:45.385558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a536f-cc42-4fbf-b858-72cba256f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:45.386203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a536f-cc42-4fbf-b858-72cba256f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:45.387063 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a536f-cc42-4fbf-b858-72cba256f9a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:45.392439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:45.405919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a536f-cc42-4fbf-b858-72cba256f9a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:45.408787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a536f-cc42-4fbf-b858-72cba256f9a2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-162a536f-cc42-4fbf-b858-72cba256f9a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:45.439469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:45.440634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:45.441713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:45.441713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:45.441713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:45.441713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:45.441713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:45.441713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:45.441713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:45.441713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:45.441713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:45.441713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:45.443650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 345/690] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 06:17:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:45.449039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a536f-cc42-4fbf-b858-72cba256f9a2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:45.558562 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055dbda-a96d-4ad2-9b30-404224b14d8b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:45.558939 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a536f-cc42-4fbf-b858-72cba256f9a2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:45.597820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347793df-c89e-4806-81c9-905e25390d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:45.599191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347793df-c89e-4806-81c9-905e25390d87 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:45.599891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347793df-c89e-4806-81c9-905e25390d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:45.600322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347793df-c89e-4806-81c9-905e25390d87 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:45.601604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347793df-c89e-4806-81c9-905e25390d87 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:45.602398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347793df-c89e-4806-81c9-905e25390d87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:45.603977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347793df-c89e-4806-81c9-905e25390d87 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:45.614075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0abe3-d80a-4eee-b61a-d2a8050a71de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:45.614898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0abe3-d80a-4eee-b61a-d2a8050a71de None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:45.615288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0abe3-d80a-4eee-b61a-d2a8050a71de None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:45.615668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0abe3-d80a-4eee-b61a-d2a8050a71de None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:45.616621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0abe3-d80a-4eee-b61a-d2a8050a71de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:45.617053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0abe3-d80a-4eee-b61a-d2a8050a71de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:45.623861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 346/691] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 06:17:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:45.625363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0abe3-d80a-4eee-b61a-d2a8050a71de None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:45.631346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347793df-c89e-4806-81c9-905e25390d87 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:45.663075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 346/692] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 06:17:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:45.668630 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0abe3-d80a-4eee-b61a-d2a8050a71de None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:45.673772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:45.798310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:45.815404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0abe3-d80a-4eee-b61a-d2a8050a71de None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:45.842360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347793df-c89e-4806-81c9-905e25390d87 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:45.852735 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b6b85-0eed-41d5-9e90-3018a8ce9a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:45.853712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b6b85-0eed-41d5-9e90-3018a8ce9a38 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:45.854292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b6b85-0eed-41d5-9e90-3018a8ce9a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:45.854791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b6b85-0eed-41d5-9e90-3018a8ce9a38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:45.934543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b6b85-0eed-41d5-9e90-3018a8ce9a38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:45.937224 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b6b85-0eed-41d5-9e90-3018a8ce9a38 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:45.983646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b89ee0-ee41-4b46-a69b-f85ce8b346ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:45.986188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b89ee0-ee41-4b46-a69b-f85ce8b346ed None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:45.987307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b89ee0-ee41-4b46-a69b-f85ce8b346ed None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:45.988913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b89ee0-ee41-4b46-a69b-f85ce8b346ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.078291 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:46.108702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 347/693] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 06:17:46.112481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b6b85-0eed-41d5-9e90-3018a8ce9a38 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:46.136394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b89ee0-ee41-4b46-a69b-f85ce8b346ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:46.143466 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b89ee0-ee41-4b46-a69b-f85ce8b346ed None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:46.161964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:46.248014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b6b85-0eed-41d5-9e90-3018a8ce9a38 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:46.251695 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 347/694] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 06:17:46.252946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f758a-0368-46ac-95b7-eef0b6e18101 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:46.253305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f758a-0368-46ac-95b7-eef0b6e18101 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:46.253655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f758a-0368-46ac-95b7-eef0b6e18101 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.255141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 348/695] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:46.256142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11116e9-e173-4419-822c-28f2407a3632 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:46.256523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11116e9-e173-4419-822c-28f2407a3632 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:46.256873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11116e9-e173-4419-822c-28f2407a3632 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.258427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 349/696] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:46.260413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11116e9-e173-4419-822c-28f2407a3632 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:46.295327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb293142-338b-4ad1-be05-2d3b44d01f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:46.296121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb293142-338b-4ad1-be05-2d3b44d01f00 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:46.296406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb293142-338b-4ad1-be05-2d3b44d01f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:46.296697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb293142-338b-4ad1-be05-2d3b44d01f00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.307276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb293142-338b-4ad1-be05-2d3b44d01f00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:46.308761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb293142-338b-4ad1-be05-2d3b44d01f00 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-cb293142-338b-4ad1-be05-2d3b44d01f00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:46.327731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:46.329251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:46.330415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:46.330415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:46.330415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:46.330415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:46.333570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 348/697] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:46.334889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a0868-ed44-45ae-91bd-750924758408 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:46.335176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a0868-ed44-45ae-91bd-750924758408 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:46.335458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a0868-ed44-45ae-91bd-750924758408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.353083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-446a0868-ed44-45ae-91bd-750924758408 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:46.354298 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-446a0868-ed44-45ae-91bd-750924758408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:46.375667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11116e9-e173-4419-822c-28f2407a3632 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:46.385413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:46.396054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:46.416857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc053535-1b00-46a4-8268-2e1ae0b91553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:46.417925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc053535-1b00-46a4-8268-2e1ae0b91553 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:46.418239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc053535-1b00-46a4-8268-2e1ae0b91553 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:46.418561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc053535-1b00-46a4-8268-2e1ae0b91553 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.437964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 349/698] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:46.437964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0ffde-f6b3-45aa-872b-38b71e600238 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:46.437964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0ffde-f6b3-45aa-872b-38b71e600238 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:46.437964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0ffde-f6b3-45aa-872b-38b71e600238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.438382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc053535-1b00-46a4-8268-2e1ae0b91553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:46.438382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc053535-1b00-46a4-8268-2e1ae0b91553 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:46.456274 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-64c0ffde-f6b3-45aa-872b-38b71e600238 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:46.457455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-64c0ffde-f6b3-45aa-872b-38b71e600238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fc053535-1b00-46a4-8268-2e1ae0b91553 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:46.457762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:46.458648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:46.459509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:46.459509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:46.459509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:46.459509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:46.463684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 350/699] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:46.464365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc053535-1b00-46a4-8268-2e1ae0b91553 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:46.533153 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 350/700] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:46.534428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0ffde-f6b3-45aa-872b-38b71e600238 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:46.578541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc053535-1b00-46a4-8268-2e1ae0b91553 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:46.615353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa79c32-5cd2-4a81-a3a4-5856a43377fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:46.616362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa79c32-5cd2-4a81-a3a4-5856a43377fe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:46.616832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa79c32-5cd2-4a81-a3a4-5856a43377fe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:46.617487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa79c32-5cd2-4a81-a3a4-5856a43377fe None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.618330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa79c32-5cd2-4a81-a3a4-5856a43377fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:46.618692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa79c32-5cd2-4a81-a3a4-5856a43377fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:46.620215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa79c32-5cd2-4a81-a3a4-5856a43377fe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:46.637375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 351/701] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:46.637375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa79c32-5cd2-4a81-a3a4-5856a43377fe None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:46.645020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:46.650277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0ffde-f6b3-45aa-872b-38b71e600238 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:46.664571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:46.688723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0eb13-1a6d-407d-b123-985be1f8b075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:46.689520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0eb13-1a6d-407d-b123-985be1f8b075 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:46.689797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0eb13-1a6d-407d-b123-985be1f8b075 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:46.690102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0eb13-1a6d-407d-b123-985be1f8b075 None admin] PATH_INFO: `/v3/services/network` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.690758 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0eb13-1a6d-407d-b123-985be1f8b075 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:46.692117 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0eb13-1a6d-407d-b123-985be1f8b075 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8bb0eb13-1a6d-407d-b123-985be1f8b075 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:46.706030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:46.707089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:46.707951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 351/702] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:46 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:46.708750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0eb13-1a6d-407d-b123-985be1f8b075 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:46.749955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa79c32-5cd2-4a81-a3a4-5856a43377fe None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:46.786343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e33998-7e5c-41b1-9b2b-b93b760075a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:46.786924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e33998-7e5c-41b1-9b2b-b93b760075a1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:46.787209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e33998-7e5c-41b1-9b2b-b93b760075a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:46.787482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e33998-7e5c-41b1-9b2b-b93b760075a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.788207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e33998-7e5c-41b1-9b2b-b93b760075a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:46.788512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e33998-7e5c-41b1-9b2b-b93b760075a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:46.789856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e33998-7e5c-41b1-9b2b-b93b760075a1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:46.807711 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 352/703] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:46.807711 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e33998-7e5c-41b1-9b2b-b93b760075a1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:46.824952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0eb13-1a6d-407d-b123-985be1f8b075 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:46.860861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df65f7a-324f-4bfb-b1bb-a984bf644661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:46.861681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df65f7a-324f-4bfb-b1bb-a984bf644661 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:46.861997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df65f7a-324f-4bfb-b1bb-a984bf644661 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:46.862282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df65f7a-324f-4bfb-b1bb-a984bf644661 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.862946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df65f7a-324f-4bfb-b1bb-a984bf644661 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:46.864343 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df65f7a-324f-4bfb-b1bb-a984bf644661 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5df65f7a-324f-4bfb-b1bb-a984bf644661 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:46.876971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:46.878096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 06:17:46.878943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:46.883117 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 352/704] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Jul 21 06:17:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:46.883758 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df65f7a-324f-4bfb-b1bb-a984bf644661 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:46.903161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:46.914040 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:46.918679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e33998-7e5c-41b1-9b2b-b93b760075a1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:46.954678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2692f-eaaf-4503-ab88-0bb89640a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:46.955467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2692f-eaaf-4503-ab88-0bb89640a8ec None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:46.955755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2692f-eaaf-4503-ab88-0bb89640a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:46.956030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2692f-eaaf-4503-ab88-0bb89640a8ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:46.962761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2692f-eaaf-4503-ab88-0bb89640a8ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:46.964194 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2692f-eaaf-4503-ab88-0bb89640a8ec None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-cec2692f-eaaf-4503-ab88-0bb89640a8ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:46.979273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:46.980205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:46.981138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:46.981138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:46.981138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:46.981138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:46.981138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:46.981138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:46.981138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:46.981138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:46.981138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 353/705] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:46.981699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2692f-eaaf-4503-ab88-0bb89640a8ec None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:46.994353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df65f7a-324f-4bfb-b1bb-a984bf644661 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:47.034486 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e837a-2d84-4c1a-9a44-e38fcce27956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:47.035148 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e837a-2d84-4c1a-9a44-e38fcce27956 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:47.035611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e837a-2d84-4c1a-9a44-e38fcce27956 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:47.035895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e837a-2d84-4c1a-9a44-e38fcce27956 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:47.036655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e837a-2d84-4c1a-9a44-e38fcce27956 None admin] RBAC: Adding query filter params (name=network) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:47.036912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e837a-2d84-4c1a-9a44-e38fcce27956 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:47.038310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e837a-2d84-4c1a-9a44-e38fcce27956 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:47.048455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 353/706] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:17:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:47.048455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e837a-2d84-4c1a-9a44-e38fcce27956 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:47.100000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2692f-eaaf-4503-ab88-0bb89640a8ec None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:47.136457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb8ecb-fbf5-45c2-b948-7173c2da96a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:47.137267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb8ecb-fbf5-45c2-b948-7173c2da96a8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:47.137585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb8ecb-fbf5-45c2-b948-7173c2da96a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:47.137892 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb8ecb-fbf5-45c2-b948-7173c2da96a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:47.145183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb8ecb-fbf5-45c2-b948-7173c2da96a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:47.146668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb8ecb-fbf5-45c2-b948-7173c2da96a8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:47.147400 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:47.159278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e837a-2d84-4c1a-9a44-e38fcce27956 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c4eb8ecb-fbf5-45c2-b948-7173c2da96a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:47.162595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:47.163504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:47.164410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:47.164410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:47.164410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:47.164410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:47.164410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:47.164410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:47.164410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:47.164410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:47.168471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 354/707] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:47.170006 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb8ecb-fbf5-45c2-b948-7173c2da96a8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:47.182104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:47.200103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510597e-f17d-423d-9d6c-8a789cfdc11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:47.201488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510597e-f17d-423d-9d6c-8a789cfdc11a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:47.201780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510597e-f17d-423d-9d6c-8a789cfdc11a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:47.202096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510597e-f17d-423d-9d6c-8a789cfdc11a None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:47.202816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510597e-f17d-423d-9d6c-8a789cfdc11a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:47.203134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510597e-f17d-423d-9d6c-8a789cfdc11a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:47.204535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510597e-f17d-423d-9d6c-8a789cfdc11a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:47.213439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 354/708] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 06:17:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:47.213439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510597e-f17d-423d-9d6c-8a789cfdc11a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:47.278448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb8ecb-fbf5-45c2-b948-7173c2da96a8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:47.314491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ae8ea-78fb-47f5-88eb-6532829eb624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:47.315080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ae8ea-78fb-47f5-88eb-6532829eb624 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:47.315391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ae8ea-78fb-47f5-88eb-6532829eb624 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:47.315674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ae8ea-78fb-47f5-88eb-6532829eb624 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:47.315801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510597e-f17d-423d-9d6c-8a789cfdc11a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:47.316468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ae8ea-78fb-47f5-88eb-6532829eb624 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:47.316781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ae8ea-78fb-47f5-88eb-6532829eb624 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:47.318260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ae8ea-78fb-47f5-88eb-6532829eb624 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:47.329714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 355/709] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:47.330541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ae8ea-78fb-47f5-88eb-6532829eb624 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:47.349916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810921f-a1cc-4c73-a3e8-611c7ff5ec9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:47.350622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810921f-a1cc-4c73-a3e8-611c7ff5ec9e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:47.350910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810921f-a1cc-4c73-a3e8-611c7ff5ec9e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:47.351195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810921f-a1cc-4c73-a3e8-611c7ff5ec9e None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:47.351908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810921f-a1cc-4c73-a3e8-611c7ff5ec9e None admin] RBAC: Adding query filter params (type=network) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:47.352219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810921f-a1cc-4c73-a3e8-611c7ff5ec9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:47.353594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810921f-a1cc-4c73-a3e8-611c7ff5ec9e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:47.359752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 355/710] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:17:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:47.362809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810921f-a1cc-4c73-a3e8-611c7ff5ec9e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:47.386216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:47.402662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:47.447322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ae8ea-78fb-47f5-88eb-6532829eb624 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:47.486305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff8706-e9f3-4dec-99d9-e85e7cebf1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:47.486991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff8706-e9f3-4dec-99d9-e85e7cebf1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:47.487259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff8706-e9f3-4dec-99d9-e85e7cebf1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:47.487535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff8706-e9f3-4dec-99d9-e85e7cebf1a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:47.488333 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff8706-e9f3-4dec-99d9-e85e7cebf1a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:47.488643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff8706-e9f3-4dec-99d9-e85e7cebf1a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:47.488808 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810921f-a1cc-4c73-a3e8-611c7ff5ec9e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:47.490119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff8706-e9f3-4dec-99d9-e85e7cebf1a3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:47.508800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 356/711] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:47.508800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82dcf0-ab16-4793-a8d3-335b396df8ee None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:47.508800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82dcf0-ab16-4793-a8d3-335b396df8ee None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:47.508800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82dcf0-ab16-4793-a8d3-335b396df8ee None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:47.508800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 357/712] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:47.508800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82dcf0-ab16-4793-a8d3-335b396df8ee None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:47.537319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3b0d4-b743-466e-a903-aab0bc4e00db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:47.538010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3b0d4-b743-466e-a903-aab0bc4e00db None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:47.538297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3b0d4-b743-466e-a903-aab0bc4e00db None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:47.538569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3b0d4-b743-466e-a903-aab0bc4e00db None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:47.539276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3b0d4-b743-466e-a903-aab0bc4e00db None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:47.539583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3b0d4-b743-466e-a903-aab0bc4e00db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:47.540998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3b0d4-b743-466e-a903-aab0bc4e00db None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:47.546826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 356/713] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 06:17:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:47.547654 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3b0d4-b743-466e-a903-aab0bc4e00db None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:47.646956 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82dcf0-ab16-4793-a8d3-335b396df8ee None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:47.650011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:47.672822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3b0d4-b743-466e-a903-aab0bc4e00db None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:47.682211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:47.692105 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4e9b0-298c-459c-ac34-447782cfdb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:47.693087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4e9b0-298c-459c-ac34-447782cfdb02 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:47.693603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4e9b0-298c-459c-ac34-447782cfdb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:47.694118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4e9b0-298c-459c-ac34-447782cfdb02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:47.695220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4e9b0-298c-459c-ac34-447782cfdb02 None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=94c8eb61f2ca48288cba81d280a30747) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:47.695709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4e9b0-298c-459c-ac34-447782cfdb02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:47.697454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4e9b0-298c-459c-ac34-447782cfdb02 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:47.704677 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 358/714] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:47 2022] GET /identity/v3/role_assignments?user.id=94c8eb61f2ca48288cba81d280a30747&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:47.706308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38277c00-7ef7-4d77-8abc-b3c4476bb006 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:47.706799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38277c00-7ef7-4d77-8abc-b3c4476bb006 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:47.707253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38277c00-7ef7-4d77-8abc-b3c4476bb006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:47.713976 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c443c-e523-4070-9070-5355884e96a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:47.714824 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c443c-e523-4070-9070-5355884e96a4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:47.714824 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c443c-e523-4070-9070-5355884e96a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:47.715018 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c443c-e523-4070-9070-5355884e96a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:47.715867 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c443c-e523-4070-9070-5355884e96a4 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=058f099e6f7545a1b2734dbe5afd2027) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:47.716863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c443c-e523-4070-9070-5355884e96a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:47.718278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c443c-e523-4070-9070-5355884e96a4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:47.726333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 357/715] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:47 2022] GET /identity/v3/role_assignments?user.id=058f099e6f7545a1b2734dbe5afd2027&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:47.728791 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-38277c00-7ef7-4d77-8abc-b3c4476bb006 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:47.730293 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-38277c00-7ef7-4d77-8abc-b3c4476bb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:47.817830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 359/716] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:47.846887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c443c-e523-4070-9070-5355884e96a4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:47.974750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c443c-e523-4070-9070-5355884e96a4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:48.011866 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5d81f-a1ec-4bc1-98f0-939234f51753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:48.014049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5d81f-a1ec-4bc1-98f0-939234f51753 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:48.014546 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5d81f-a1ec-4bc1-98f0-939234f51753 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:48.015023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5d81f-a1ec-4bc1-98f0-939234f51753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:48.027884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5d81f-a1ec-4bc1-98f0-939234f51753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:48.030551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5d81f-a1ec-4bc1-98f0-939234f51753 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:48.037623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ffb5d81f-a1ec-4bc1-98f0-939234f51753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:48.050369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:48.051493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:48.052591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:48.052591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:48.052591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:48.052591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:48.056087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 358/717] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:48.060615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38277c00-7ef7-4d77-8abc-b3c4476bb006 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:48.167758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38277c00-7ef7-4d77-8abc-b3c4476bb006 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:48.193209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:48.207698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a93ccf-4712-41f3-9a90-def5f7a51ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:48.208448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a93ccf-4712-41f3-9a90-def5f7a51ccd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:48.208731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a93ccf-4712-41f3-9a90-def5f7a51ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:48.209010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a93ccf-4712-41f3-9a90-def5f7a51ccd None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:48.209797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a93ccf-4712-41f3-9a90-def5f7a51ccd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:48.210139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a93ccf-4712-41f3-9a90-def5f7a51ccd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:48.211513 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a93ccf-4712-41f3-9a90-def5f7a51ccd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:48.226054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 360/718] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:48.231452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5d81f-a1ec-4bc1-98f0-939234f51753 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:48.342826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5d81f-a1ec-4bc1-98f0-939234f51753 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:48.387444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29165d53-0f40-476f-8617-8ab9ba50ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:48.389644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29165d53-0f40-476f-8617-8ab9ba50ef3f None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:48.390208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29165d53-0f40-476f-8617-8ab9ba50ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:48.390699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29165d53-0f40-476f-8617-8ab9ba50ef3f None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:48.391660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29165d53-0f40-476f-8617-8ab9ba50ef3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:48.393354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29165d53-0f40-476f-8617-8ab9ba50ef3f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:48.433742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 359/719] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:17:48 2022] POST /identity/v3/users => generated 254 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:17:49.593273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6b2d5-81db-46d8-835d-5f497a9c4f9e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:49.594325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6b2d5-81db-46d8-835d-5f497a9c4f9e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:49.594835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6b2d5-81db-46d8-835d-5f497a9c4f9e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:49.596669 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 361/720] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:49.601787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170f409-0a4a-46e2-9de2-c0f2ad3198a3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:49.602617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170f409-0a4a-46e2-9de2-c0f2ad3198a3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:49.603101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170f409-0a4a-46e2-9de2-c0f2ad3198a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:49.620924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3170f409-0a4a-46e2-9de2-c0f2ad3198a3 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:49.623048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3170f409-0a4a-46e2-9de2-c0f2ad3198a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:49.684263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:49.710515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 360/721] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 06:17:49.751368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6b2d5-81db-46d8-835d-5f497a9c4f9e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:49.887771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6b2d5-81db-46d8-835d-5f497a9c4f9e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:49.940338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b974-b417-47e1-98dc-d7cd2e443080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:49.941030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b974-b417-47e1-98dc-d7cd2e443080 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:49.941316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b974-b417-47e1-98dc-d7cd2e443080 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:49.941598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b974-b417-47e1-98dc-d7cd2e443080 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:49.942425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7b974-b417-47e1-98dc-d7cd2e443080 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:49.943797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7b974-b417-47e1-98dc-d7cd2e443080 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:49.974606 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 362/722] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:17:49 2022] POST /identity/v3/services => generated 240 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:17:50.089600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9ae51-8ae7-45e9-8d47-10c65907fb0f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:50.089919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9ae51-8ae7-45e9-8d47-10c65907fb0f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:50.090195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9ae51-8ae7-45e9-8d47-10c65907fb0f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:50.092652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 361/723] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:50.098352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13762b45-0040-4600-b25f-ca897e75935a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:50.098975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13762b45-0040-4600-b25f-ca897e75935a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:50.099486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13762b45-0040-4600-b25f-ca897e75935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:50.125883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-13762b45-0040-4600-b25f-ca897e75935a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:50.127464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-13762b45-0040-4600-b25f-ca897e75935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:50.183208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:50.242967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 363/724] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:50 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 06:17:50.266587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9ae51-8ae7-45e9-8d47-10c65907fb0f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:50.371226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a99a57-c4b5-43bf-938a-c5a1fa929400 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:50.371565 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a99a57-c4b5-43bf-938a-c5a1fa929400 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:50.371801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a99a57-c4b5-43bf-938a-c5a1fa929400 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:50.373221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 364/725] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:50.380788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01506358-6e4b-4c13-99ee-20d255f3c30a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:50.381087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01506358-6e4b-4c13-99ee-20d255f3c30a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:50.381375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01506358-6e4b-4c13-99ee-20d255f3c30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:50.393628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9ae51-8ae7-45e9-8d47-10c65907fb0f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:50.404081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-01506358-6e4b-4c13-99ee-20d255f3c30a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:50.405629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-01506358-6e4b-4c13-99ee-20d255f3c30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:50.434482 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13f637-5fa0-436d-bd55-b7363ef3d025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:50.435090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13f637-5fa0-436d-bd55-b7363ef3d025 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:50.435373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13f637-5fa0-436d-bd55-b7363ef3d025 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:50.435656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13f637-5fa0-436d-bd55-b7363ef3d025 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:50.437374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13f637-5fa0-436d-bd55-b7363ef3d025 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:50.438866 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13f637-5fa0-436d-bd55-b7363ef3d025 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:50.466328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 362/726] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:17:50 2022] POST /identity/v3/services => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:17:50.499045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab208c7f-b23b-4d72-b6aa-4d2b6462252f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:50.499334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab208c7f-b23b-4d72-b6aa-4d2b6462252f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:50.499624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab208c7f-b23b-4d72-b6aa-4d2b6462252f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:50.501356 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 363/727] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:17:50.506794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 365/728] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:17:50.507019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43136c8-413e-47b4-82c9-259c8c22a3f6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:50.507332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43136c8-413e-47b4-82c9-259c8c22a3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:50.507677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43136c8-413e-47b4-82c9-259c8c22a3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:50.534497 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01506358-6e4b-4c13-99ee-20d255f3c30a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:50.539318 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f43136c8-413e-47b4-82c9-259c8c22a3f6 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:50.540890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f43136c8-413e-47b4-82c9-259c8c22a3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:50.595115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:50.627891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 364/729] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:17:50.666800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:50.678800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01506358-6e4b-4c13-99ee-20d255f3c30a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:50.692200 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43136c8-413e-47b4-82c9-259c8c22a3f6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:50.719549 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bffab7-2c3e-465b-8bc9-f68f1606d971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:50.720414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bffab7-2c3e-465b-8bc9-f68f1606d971 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:50.720945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bffab7-2c3e-465b-8bc9-f68f1606d971 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:50.721288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bffab7-2c3e-465b-8bc9-f68f1606d971 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:50.737931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bffab7-2c3e-465b-8bc9-f68f1606d971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a1bffab7-2c3e-465b-8bc9-f68f1606d971 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:50.748444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:50.749402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:50.750380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 366/730] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:50.758175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bffab7-2c3e-465b-8bc9-f68f1606d971 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:50.829803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43136c8-413e-47b4-82c9-259c8c22a3f6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:50.865746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e024c-35da-459e-bdfe-f02f05ccf117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:50.867375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e024c-35da-459e-bdfe-f02f05ccf117 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:50.868021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e024c-35da-459e-bdfe-f02f05ccf117 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:50.868610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e024c-35da-459e-bdfe-f02f05ccf117 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:50.882256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bffab7-2c3e-465b-8bc9-f68f1606d971 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:50.883009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e024c-35da-459e-bdfe-f02f05ccf117 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8e7e024c-35da-459e-bdfe-f02f05ccf117 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:50.893775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:50.894744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:50.895578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:50.899293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 365/731] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:50.905677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e024c-35da-459e-bdfe-f02f05ccf117 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:50.924107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d281f12-0a53-41cc-9ed7-2ad29e30c4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:50.924732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d281f12-0a53-41cc-9ed7-2ad29e30c4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:50.925022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d281f12-0a53-41cc-9ed7-2ad29e30c4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:50.925298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d281f12-0a53-41cc-9ed7-2ad29e30c4a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:50.926088 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d281f12-0a53-41cc-9ed7-2ad29e30c4a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:50.926398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d281f12-0a53-41cc-9ed7-2ad29e30c4a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:50.927893 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d281f12-0a53-41cc-9ed7-2ad29e30c4a5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:50.937677 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 367/732] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:50.943581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d281f12-0a53-41cc-9ed7-2ad29e30c4a5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:50.966633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:50.972705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:51.058053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d281f12-0a53-41cc-9ed7-2ad29e30c4a5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:51.096227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08106e4-51a5-4630-9e7f-6ec879a0730c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:51.097561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08106e4-51a5-4630-9e7f-6ec879a0730c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:51.098150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08106e4-51a5-4630-9e7f-6ec879a0730c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:51.098628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08106e4-51a5-4630-9e7f-6ec879a0730c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:51.110976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08106e4-51a5-4630-9e7f-6ec879a0730c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:51.112717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08106e4-51a5-4630-9e7f-6ec879a0730c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e08106e4-51a5-4630-9e7f-6ec879a0730c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:51.132213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:51.133192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:51.134300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:51.134300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:51.134300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:51.134300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:51.136576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 368/733] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:51.142925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08106e4-51a5-4630-9e7f-6ec879a0730c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:51.230783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:51.253339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08106e4-51a5-4630-9e7f-6ec879a0730c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:51.259901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e024c-35da-459e-bdfe-f02f05ccf117 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:51.294379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a6d55-cebd-4431-b74c-de4fb2be1727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:51.295168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6d55-cebd-4431-b74c-de4fb2be1727 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:51.295535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6d55-cebd-4431-b74c-de4fb2be1727 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:51.295902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6d55-cebd-4431-b74c-de4fb2be1727 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:51.296720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a6d55-cebd-4431-b74c-de4fb2be1727 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:51.297109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a6d55-cebd-4431-b74c-de4fb2be1727 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:51.298670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a6d55-cebd-4431-b74c-de4fb2be1727 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:51.311867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 369/734] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:51.317458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a6d55-cebd-4431-b74c-de4fb2be1727 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:51.362009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcfef4-c6bb-4c6f-9c33-cb31c4a66841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:51.362730 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcfef4-c6bb-4c6f-9c33-cb31c4a66841 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:51.362963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcfef4-c6bb-4c6f-9c33-cb31c4a66841 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:51.363238 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcfef4-c6bb-4c6f-9c33-cb31c4a66841 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:51.363994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcfef4-c6bb-4c6f-9c33-cb31c4a66841 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:51.364301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcfef4-c6bb-4c6f-9c33-cb31c4a66841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:51.365711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcfef4-c6bb-4c6f-9c33-cb31c4a66841 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:51.385390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 366/735] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:51.391366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcfef4-c6bb-4c6f-9c33-cb31c4a66841 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:51.427289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a6d55-cebd-4431-b74c-de4fb2be1727 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:51.456925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:51.466513 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a49efc-d64a-4948-ad47-717c8fbaba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:51.468354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49efc-d64a-4948-ad47-717c8fbaba5b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:51.468903 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49efc-d64a-4948-ad47-717c8fbaba5b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:51.469327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49efc-d64a-4948-ad47-717c8fbaba5b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:51.478822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a49efc-d64a-4948-ad47-717c8fbaba5b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:51.481139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a49efc-d64a-4948-ad47-717c8fbaba5b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-86a49efc-d64a-4948-ad47-717c8fbaba5b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:51.500135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:51.501313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:51.502487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:51.502487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:51.502487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:51.502487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:51.502487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:51.502487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:51.502487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:51.502487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:51.502487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:51.502487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:51.505737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 370/736] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 06:17:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:51.510957 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee041e44-f4d8-444d-8856-a7dd8d045538 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:51.511099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee041e44-f4d8-444d-8856-a7dd8d045538 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:51.511376 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee041e44-f4d8-444d-8856-a7dd8d045538 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:51.513419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 371/737] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:51.514737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee041e44-f4d8-444d-8856-a7dd8d045538 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:51.544686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:51.552230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcfef4-c6bb-4c6f-9c33-cb31c4a66841 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:51.591833 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176e1e3-b0dd-467b-8e42-812c1804deec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:51.592648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176e1e3-b0dd-467b-8e42-812c1804deec None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:51.592942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176e1e3-b0dd-467b-8e42-812c1804deec None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:51.593225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176e1e3-b0dd-467b-8e42-812c1804deec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:51.608091 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176e1e3-b0dd-467b-8e42-812c1804deec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:51.610510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176e1e3-b0dd-467b-8e42-812c1804deec None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e176e1e3-b0dd-467b-8e42-812c1804deec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:51.631762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:51.633105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:51.634517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:51.634517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:51.634517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:51.634517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:51.636483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 367/738] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:51.638488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1481d-f152-4908-801f-879687ef3b8c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:51.639001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1481d-f152-4908-801f-879687ef3b8c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:51.639478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1481d-f152-4908-801f-879687ef3b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:51.644657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee041e44-f4d8-444d-8856-a7dd8d045538 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:51.657617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-89f1481d-f152-4908-801f-879687ef3b8c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:51.659229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-89f1481d-f152-4908-801f-879687ef3b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:51.681449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6755bb6-79d6-4743-b632-07f714eea8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:51.682258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6755bb6-79d6-4743-b632-07f714eea8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:51.682527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6755bb6-79d6-4743-b632-07f714eea8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:51.682821 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6755bb6-79d6-4743-b632-07f714eea8a0 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:51.683722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6755bb6-79d6-4743-b632-07f714eea8a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:51.684036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6755bb6-79d6-4743-b632-07f714eea8a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:51.686575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6755bb6-79d6-4743-b632-07f714eea8a0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:51.711472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 372/739] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 06:17:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:51.711472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6755bb6-79d6-4743-b632-07f714eea8a0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:51.746251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 368/740] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:17:51.747999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1481d-f152-4908-801f-879687ef3b8c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:51.765964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:51.823024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6755bb6-79d6-4743-b632-07f714eea8a0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:51.846028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:51.863732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd93c5a-9857-47bd-887c-6a562d175ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:51.864750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd93c5a-9857-47bd-887c-6a562d175ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:51.865221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd93c5a-9857-47bd-887c-6a562d175ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:51.865694 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd93c5a-9857-47bd-887c-6a562d175ac3 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:51.866636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd93c5a-9857-47bd-887c-6a562d175ac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:51.867114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd93c5a-9857-47bd-887c-6a562d175ac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:51.868954 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd93c5a-9857-47bd-887c-6a562d175ac3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:51.871762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1481d-f152-4908-801f-879687ef3b8c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:51.882654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 373/741] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:51.883898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd93c5a-9857-47bd-887c-6a562d175ac3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:51.911054 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b89fe4-f706-4ee8-a1ea-ed8e5685ebd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:51.912110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b89fe4-f706-4ee8-a1ea-ed8e5685ebd9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:51.912495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b89fe4-f706-4ee8-a1ea-ed8e5685ebd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:51.912875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b89fe4-f706-4ee8-a1ea-ed8e5685ebd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:51.925033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b89fe4-f706-4ee8-a1ea-ed8e5685ebd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:51.928102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b89fe4-f706-4ee8-a1ea-ed8e5685ebd9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-63b89fe4-f706-4ee8-a1ea-ed8e5685ebd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:51.951688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:51.952865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:51.953807 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:51.953807 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:51.953807 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:51.953807 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:51.959003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 369/742] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:51.960071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b89fe4-f706-4ee8-a1ea-ed8e5685ebd9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:51.995612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd93c5a-9857-47bd-887c-6a562d175ac3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:52.003591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:52.033429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb22319-c44d-4d71-8fe4-13309afbe87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:52.034402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb22319-c44d-4d71-8fe4-13309afbe87b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:52.034694 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb22319-c44d-4d71-8fe4-13309afbe87b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:52.034977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb22319-c44d-4d71-8fe4-13309afbe87b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:52.049800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb22319-c44d-4d71-8fe4-13309afbe87b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-afb22319-c44d-4d71-8fe4-13309afbe87b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:52.061986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:52.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:52.063913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 374/743] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:52.064630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb22319-c44d-4d71-8fe4-13309afbe87b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:52.082616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b89fe4-f706-4ee8-a1ea-ed8e5685ebd9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:52.117430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:52.125532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5819d5a-58e7-47ec-b9f9-85da4dd2b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:52.126677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5819d5a-58e7-47ec-b9f9-85da4dd2b29e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:52.127169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5819d5a-58e7-47ec-b9f9-85da4dd2b29e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:52.127650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5819d5a-58e7-47ec-b9f9-85da4dd2b29e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:52.135435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5819d5a-58e7-47ec-b9f9-85da4dd2b29e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:52.137124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5819d5a-58e7-47ec-b9f9-85da4dd2b29e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f5819d5a-58e7-47ec-b9f9-85da4dd2b29e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:52.154377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:52.155326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:52.156289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:52.156289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:52.156289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:52.156289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:52.156289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:52.156289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:52.156289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:52.156289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:52.156289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:17:52.156289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:52.158929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 370/744] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:52.160007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5819d5a-58e7-47ec-b9f9-85da4dd2b29e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:52.190734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb22319-c44d-4d71-8fe4-13309afbe87b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:52.236946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce6ff3-b137-4d87-af51-ceb2a7ecceac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:52.237622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce6ff3-b137-4d87-af51-ceb2a7ecceac None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:52.237910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce6ff3-b137-4d87-af51-ceb2a7ecceac None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:52.238247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce6ff3-b137-4d87-af51-ceb2a7ecceac None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:52.238987 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce6ff3-b137-4d87-af51-ceb2a7ecceac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:52.239299 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce6ff3-b137-4d87-af51-ceb2a7ecceac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:52.241089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce6ff3-b137-4d87-af51-ceb2a7ecceac None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:52.256719 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 375/745] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:52.257665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce6ff3-b137-4d87-af51-ceb2a7ecceac None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:52.268112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5819d5a-58e7-47ec-b9f9-85da4dd2b29e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:52.293561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:52.307981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a25236-d36c-47d0-b031-7850d1355bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:52.308993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a25236-d36c-47d0-b031-7850d1355bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:52.309527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a25236-d36c-47d0-b031-7850d1355bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:52.310469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a25236-d36c-47d0-b031-7850d1355bb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:52.311537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a25236-d36c-47d0-b031-7850d1355bb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:52.312071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a25236-d36c-47d0-b031-7850d1355bb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:52.313737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a25236-d36c-47d0-b031-7850d1355bb3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:52.323447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 371/746] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:52.324615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a25236-d36c-47d0-b031-7850d1355bb3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:52.383363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:52.385001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce6ff3-b137-4d87-af51-ceb2a7ecceac None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:52.429799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f518833-1d98-4868-a935-29a25b81f1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:52.430770 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518833-1d98-4868-a935-29a25b81f1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:52.431059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518833-1d98-4868-a935-29a25b81f1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:52.431343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518833-1d98-4868-a935-29a25b81f1e6 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:52.432353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f518833-1d98-4868-a935-29a25b81f1e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:52.432685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f518833-1d98-4868-a935-29a25b81f1e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:52.434796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f518833-1d98-4868-a935-29a25b81f1e6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:52.439842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a25236-d36c-47d0-b031-7850d1355bb3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:52.458396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 376/747] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 06:17:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:52.459635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f518833-1d98-4868-a935-29a25b81f1e6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:52.489398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7ad46-0820-4243-86be-bbb57ac38fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:52.490841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7ad46-0820-4243-86be-bbb57ac38fda None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:52.491133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7ad46-0820-4243-86be-bbb57ac38fda None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:52.491416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7ad46-0820-4243-86be-bbb57ac38fda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:52.503079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7ad46-0820-4243-86be-bbb57ac38fda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:52.505001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7ad46-0820-4243-86be-bbb57ac38fda None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-52a7ad46-0820-4243-86be-bbb57ac38fda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:52.525090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:52.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:52.527547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:52.527547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:52.527547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:52.527547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:52.527547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:52.527547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:52.527547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:52.527547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:52.527547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 372/748] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:52.527547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7ad46-0820-4243-86be-bbb57ac38fda None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:52.557210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:52.582019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f518833-1d98-4868-a935-29a25b81f1e6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:52.620600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26da7c1-3521-481d-a086-12d983a92b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:52.621884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26da7c1-3521-481d-a086-12d983a92b39 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:52.622705 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26da7c1-3521-481d-a086-12d983a92b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:52.623285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26da7c1-3521-481d-a086-12d983a92b39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:52.635025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26da7c1-3521-481d-a086-12d983a92b39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:52.636809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26da7c1-3521-481d-a086-12d983a92b39 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:52.652638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7ad46-0820-4243-86be-bbb57ac38fda None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a26da7c1-3521-481d-a086-12d983a92b39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:52.659325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:52.660188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:52.661058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:52.661058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:52.661058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:52.661058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:52.663495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 377/749] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:52.664675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26da7c1-3521-481d-a086-12d983a92b39 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:52.671715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:52.707418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084667aa-1a8d-4b25-8815-6cb5f51e3116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:52.709237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084667aa-1a8d-4b25-8815-6cb5f51e3116 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:52.709743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084667aa-1a8d-4b25-8815-6cb5f51e3116 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:52.710233 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084667aa-1a8d-4b25-8815-6cb5f51e3116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:52.791598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084667aa-1a8d-4b25-8815-6cb5f51e3116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:52.793086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084667aa-1a8d-4b25-8815-6cb5f51e3116 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:52.894986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26da7c1-3521-481d-a086-12d983a92b39 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:52.908206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 373/750] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:17:52.908206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084667aa-1a8d-4b25-8815-6cb5f51e3116 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:52.922876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:52.938448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3fc38-abc3-4f93-8f33-a530a93bb8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:52.939247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3fc38-abc3-4f93-8f33-a530a93bb8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:52.939637 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3fc38-abc3-4f93-8f33-a530a93bb8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:52.939985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3fc38-abc3-4f93-8f33-a530a93bb8e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:52.940783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3fc38-abc3-4f93-8f33-a530a93bb8e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:52.941089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3fc38-abc3-4f93-8f33-a530a93bb8e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:52.942550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3fc38-abc3-4f93-8f33-a530a93bb8e8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:52.945172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:52.956127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 378/751] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:52.957389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0356f-e53d-448a-8bf9-5bb236db449c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:52.957679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0356f-e53d-448a-8bf9-5bb236db449c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:52.957976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0356f-e53d-448a-8bf9-5bb236db449c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:52.961995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 379/752] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:52.961995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0356f-e53d-448a-8bf9-5bb236db449c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:53.014829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084667aa-1a8d-4b25-8815-6cb5f51e3116 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:53.048777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb4303-ee18-4b01-a544-60f5854d7c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:53.049370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4303-ee18-4b01-a544-60f5854d7c69 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.049651 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4303-ee18-4b01-a544-60f5854d7c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.049950 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4303-ee18-4b01-a544-60f5854d7c69 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.050683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb4303-ee18-4b01-a544-60f5854d7c69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:53.050987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb4303-ee18-4b01-a544-60f5854d7c69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:53.052350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb4303-ee18-4b01-a544-60f5854d7c69 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:53.063629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0356f-e53d-448a-8bf9-5bb236db449c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:53.064512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 374/753] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:53.065779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a6b69-7396-4dc7-913e-086b213f21c1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.066103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a6b69-7396-4dc7-913e-086b213f21c1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.066388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a6b69-7396-4dc7-913e-086b213f21c1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.067601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 375/754] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:53.068277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a6b69-7396-4dc7-913e-086b213f21c1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:53.102201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932a00e-af59-4a10-97b1-a4768d250a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:53.102995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932a00e-af59-4a10-97b1-a4768d250a55 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.103284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932a00e-af59-4a10-97b1-a4768d250a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.103561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932a00e-af59-4a10-97b1-a4768d250a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.113770 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932a00e-af59-4a10-97b1-a4768d250a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:53.115242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932a00e-af59-4a10-97b1-a4768d250a55 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4932a00e-af59-4a10-97b1-a4768d250a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:53.132836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:53.133904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:53.134835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:53.134835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:53.134835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:53.134835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:53.134835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 380/755] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:53.135180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67d438-4613-489a-9ab3-1e9b98f2e9d7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.135470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67d438-4613-489a-9ab3-1e9b98f2e9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.135752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67d438-4613-489a-9ab3-1e9b98f2e9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.145195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:53.152473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ff67d438-4613-489a-9ab3-1e9b98f2e9d7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:53.153919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ff67d438-4613-489a-9ab3-1e9b98f2e9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:53.176572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a6b69-7396-4dc7-913e-086b213f21c1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:53.196657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:53.210301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f92828-4db0-4271-b48c-a282401d7fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:53.211189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f92828-4db0-4271-b48c-a282401d7fdc None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.211479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f92828-4db0-4271-b48c-a282401d7fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.211764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f92828-4db0-4271-b48c-a282401d7fdc None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/058f099e6f7545a1b2734dbe5afd2027/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.213743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f92828-4db0-4271-b48c-a282401d7fdc None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=058f099e6f7545a1b2734dbe5afd2027, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:53.215220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f92828-4db0-4271-b48c-a282401d7fdc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:53.221278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 381/756] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:17:53.222142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67d438-4613-489a-9ab3-1e9b98f2e9d7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:53.243465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 376/757] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:17:53 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/058f099e6f7545a1b2734dbe5afd2027/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:17:53.243465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7a192-4e35-47b3-af9b-997e09cf3de5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.243465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7a192-4e35-47b3-af9b-997e09cf3de5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.243465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7a192-4e35-47b3-af9b-997e09cf3de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.270112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8ef7a192-4e35-47b3-af9b-997e09cf3de5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:53.271433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8ef7a192-4e35-47b3-af9b-997e09cf3de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:53.337704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67d438-4613-489a-9ab3-1e9b98f2e9d7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:53.360874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 377/758] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:17:53.360874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7a192-4e35-47b3-af9b-997e09cf3de5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:53.380691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28ccbe-54dd-4283-9b8d-e468551a5149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:53.380691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28ccbe-54dd-4283-9b8d-e468551a5149 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.380691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28ccbe-54dd-4283-9b8d-e468551a5149 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.380691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28ccbe-54dd-4283-9b8d-e468551a5149 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.393725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28ccbe-54dd-4283-9b8d-e468551a5149 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:53.395282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28ccbe-54dd-4283-9b8d-e468551a5149 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0a28ccbe-54dd-4283-9b8d-e468551a5149 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:53.412901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:53.413907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:53.415170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:53.415170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:53.415170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:53.415170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:53.415170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:53.415170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:53.415170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:53.415170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:53.415170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:17:53.415170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:53.418845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 382/759] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 06:17:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:53.419605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28ccbe-54dd-4283-9b8d-e468551a5149 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:53.440247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:53.462329 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:53.485217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7a192-4e35-47b3-af9b-997e09cf3de5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:53.522240 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b536b6d-661f-421e-90d8-fcb856d8efa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:53.523279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b536b6d-661f-421e-90d8-fcb856d8efa3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.523560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b536b6d-661f-421e-90d8-fcb856d8efa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.523831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b536b6d-661f-421e-90d8-fcb856d8efa3 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.524552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b536b6d-661f-421e-90d8-fcb856d8efa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:53.524855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b536b6d-661f-421e-90d8-fcb856d8efa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:53.525868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28ccbe-54dd-4283-9b8d-e468551a5149 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:53.526258 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b536b6d-661f-421e-90d8-fcb856d8efa3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:53.543534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 378/760] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:53.543534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b536b6d-661f-421e-90d8-fcb856d8efa3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:53.567744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecf9a6-8169-4fd3-8632-4933e5f0518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:53.568542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecf9a6-8169-4fd3-8632-4933e5f0518f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.568823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecf9a6-8169-4fd3-8632-4933e5f0518f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.569095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecf9a6-8169-4fd3-8632-4933e5f0518f None admin] PATH_INFO: `/v3/services/network` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.569743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecf9a6-8169-4fd3-8632-4933e5f0518f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:53.571123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecf9a6-8169-4fd3-8632-4933e5f0518f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-63ecf9a6-8169-4fd3-8632-4933e5f0518f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:53.585726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:53.587000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:53.588058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 383/761] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:53 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:53.588863 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecf9a6-8169-4fd3-8632-4933e5f0518f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:53.651212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b536b6d-661f-421e-90d8-fcb856d8efa3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:53.676284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:53.689922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16b3c7-cfe5-4f1d-a4dc-3fd46c068ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:53.690763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b3c7-cfe5-4f1d-a4dc-3fd46c068ebe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.691042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b3c7-cfe5-4f1d-a4dc-3fd46c068ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.691341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b3c7-cfe5-4f1d-a4dc-3fd46c068ebe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.691991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16b3c7-cfe5-4f1d-a4dc-3fd46c068ebe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:53.693756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16b3c7-cfe5-4f1d-a4dc-3fd46c068ebe None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:53.703697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecf9a6-8169-4fd3-8632-4933e5f0518f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:53.705354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8a16b3c7-cfe5-4f1d-a4dc-3fd46c068ebe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:53.706528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:53.707452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:53.708365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 379/762] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Jul 21 06:17:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:53.709549 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16b3c7-cfe5-4f1d-a4dc-3fd46c068ebe None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:53.747952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df035b70-d7b0-4c9a-8363-8d9d07db775f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:53.748632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df035b70-d7b0-4c9a-8363-8d9d07db775f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.748920 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df035b70-d7b0-4c9a-8363-8d9d07db775f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.749192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df035b70-d7b0-4c9a-8363-8d9d07db775f None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.750087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df035b70-d7b0-4c9a-8363-8d9d07db775f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:53.750470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df035b70-d7b0-4c9a-8363-8d9d07db775f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:53.751909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df035b70-d7b0-4c9a-8363-8d9d07db775f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:53.769906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 384/763] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 06:17:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:53.770767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df035b70-d7b0-4c9a-8363-8d9d07db775f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:53.819495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16b3c7-cfe5-4f1d-a4dc-3fd46c068ebe None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:53.864157 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7663cae-2b40-4ed6-bc70-bdaa480b4ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:53.864877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7663cae-2b40-4ed6-bc70-bdaa480b4ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.865679 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7663cae-2b40-4ed6-bc70-bdaa480b4ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.865988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7663cae-2b40-4ed6-bc70-bdaa480b4ca8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.876466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7663cae-2b40-4ed6-bc70-bdaa480b4ca8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:53.877930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7663cae-2b40-4ed6-bc70-bdaa480b4ca8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:53.878253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df035b70-d7b0-4c9a-8363-8d9d07db775f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c7663cae-2b40-4ed6-bc70-bdaa480b4ca8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:53.897696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:53.899035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:53.900506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:53.900506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:53.900506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:53.900506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:53.900506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:53.900506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:53.900506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:53.900506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:53.900506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 380/764] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:53.900506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7663cae-2b40-4ed6-bc70-bdaa480b4ca8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:53.917045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136563f-3e16-408e-b880-de8678034a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:53.917723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136563f-3e16-408e-b880-de8678034a10 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:53.918043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136563f-3e16-408e-b880-de8678034a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:53.918330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136563f-3e16-408e-b880-de8678034a10 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:53.919104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136563f-3e16-408e-b880-de8678034a10 None admin] RBAC: Adding query filter params (name=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:53.919420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136563f-3e16-408e-b880-de8678034a10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:53.920796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136563f-3e16-408e-b880-de8678034a10 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:53.926302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 385/765] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:17:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:53.927348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136563f-3e16-408e-b880-de8678034a10 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:53.959485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:53.965662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:54.025141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7663cae-2b40-4ed6-bc70-bdaa480b4ca8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:54.040016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136563f-3e16-408e-b880-de8678034a10 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:54.068144 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bb7dd-4219-41af-8dab-6dd9099e8ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:54.068815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bb7dd-4219-41af-8dab-6dd9099e8ece None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:54.069097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bb7dd-4219-41af-8dab-6dd9099e8ece None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:54.069373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bb7dd-4219-41af-8dab-6dd9099e8ece None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:54.070104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bb7dd-4219-41af-8dab-6dd9099e8ece None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:54.070411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bb7dd-4219-41af-8dab-6dd9099e8ece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:54.071770 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bb7dd-4219-41af-8dab-6dd9099e8ece None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:54.077978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 381/766] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 06:17:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:54.077978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bb7dd-4219-41af-8dab-6dd9099e8ece None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:54.085218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea135301-36d4-4415-91ae-5cd9fbe0e610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:54.085909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea135301-36d4-4415-91ae-5cd9fbe0e610 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:54.086229 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea135301-36d4-4415-91ae-5cd9fbe0e610 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:54.086511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea135301-36d4-4415-91ae-5cd9fbe0e610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:54.160121 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea135301-36d4-4415-91ae-5cd9fbe0e610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:54.161498 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea135301-36d4-4415-91ae-5cd9fbe0e610 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:54.179745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bb7dd-4219-41af-8dab-6dd9099e8ece None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:54.190446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:54.192487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:54.222365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaddb8-75d3-4a0a-8f9f-0764656ceb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:54.223271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaddb8-75d3-4a0a-8f9f-0764656ceb83 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:54.223383 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaddb8-75d3-4a0a-8f9f-0764656ceb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:54.223583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaddb8-75d3-4a0a-8f9f-0764656ceb83 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:54.224376 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaddb8-75d3-4a0a-8f9f-0764656ceb83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:54.224784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaddb8-75d3-4a0a-8f9f-0764656ceb83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:54.226146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaddb8-75d3-4a0a-8f9f-0764656ceb83 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:54.238511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 382/767] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:54.242832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaddb8-75d3-4a0a-8f9f-0764656ceb83 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:54.277985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 386/768] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:17:54.277985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea135301-36d4-4415-91ae-5cd9fbe0e610 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:54.361613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaddb8-75d3-4a0a-8f9f-0764656ceb83 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:54.385700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea135301-36d4-4415-91ae-5cd9fbe0e610 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:54.398248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68ff3b-5835-4899-ad86-208d3aac3943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:54.398895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68ff3b-5835-4899-ad86-208d3aac3943 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:54.399172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68ff3b-5835-4899-ad86-208d3aac3943 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:54.399445 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68ff3b-5835-4899-ad86-208d3aac3943 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:54.400150 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68ff3b-5835-4899-ad86-208d3aac3943 None admin] RBAC: Adding query filter params (type=network) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:54.400457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68ff3b-5835-4899-ad86-208d3aac3943 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:54.401819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68ff3b-5835-4899-ad86-208d3aac3943 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:54.411567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 383/769] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:17:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:54.411567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68ff3b-5835-4899-ad86-208d3aac3943 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:54.416075 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:54.429050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f985cc2-5700-4c68-b48f-a70f49618974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:54.429731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f985cc2-5700-4c68-b48f-a70f49618974 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:54.430056 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f985cc2-5700-4c68-b48f-a70f49618974 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:54.430342 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f985cc2-5700-4c68-b48f-a70f49618974 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:54.431059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f985cc2-5700-4c68-b48f-a70f49618974 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:54.431369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f985cc2-5700-4c68-b48f-a70f49618974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:54.432733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f985cc2-5700-4c68-b48f-a70f49618974 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:54.439206 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 387/770] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 06:17:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:54.439996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f985cc2-5700-4c68-b48f-a70f49618974 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:54.449469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:54.520791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68ff3b-5835-4899-ad86-208d3aac3943 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:54.546406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f985cc2-5700-4c68-b48f-a70f49618974 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:54.560571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8728e79-fe84-49b2-839f-add7230bc907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:54.561669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8728e79-fe84-49b2-839f-add7230bc907 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:54.562158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8728e79-fe84-49b2-839f-add7230bc907 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:54.562702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8728e79-fe84-49b2-839f-add7230bc907 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:54.564014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8728e79-fe84-49b2-839f-add7230bc907 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=94c8eb61f2ca48288cba81d280a30747) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:54.564516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8728e79-fe84-49b2-839f-add7230bc907 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:54.566523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8728e79-fe84-49b2-839f-add7230bc907 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:54.574047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 384/771] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:54 2022] GET /identity/v3/role_assignments?user.id=94c8eb61f2ca48288cba81d280a30747&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:54.576001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8728e79-fe84-49b2-839f-add7230bc907 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:54.585524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6bf01-85c8-4834-a24e-105582020ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:54.586469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6bf01-85c8-4834-a24e-105582020ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:54.586757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6bf01-85c8-4834-a24e-105582020ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:54.587039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6bf01-85c8-4834-a24e-105582020ae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:54.599540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6bf01-85c8-4834-a24e-105582020ae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:54.601135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6bf01-85c8-4834-a24e-105582020ae4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-99e6bf01-85c8-4834-a24e-105582020ae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:54.621117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:54.622501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:54.623693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:54.623693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:54.623693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:54.623693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:54.627230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 388/772] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:54.628162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6bf01-85c8-4834-a24e-105582020ae4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:54.676284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:54.722497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:54.728361 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8728e79-fe84-49b2-839f-add7230bc907 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:54.761605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6bf01-85c8-4834-a24e-105582020ae4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:54.782948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afc2be-4117-4dd0-9f0d-50ba1ce17e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:54.783870 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afc2be-4117-4dd0-9f0d-50ba1ce17e8b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:54.784157 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afc2be-4117-4dd0-9f0d-50ba1ce17e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:54.784433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afc2be-4117-4dd0-9f0d-50ba1ce17e8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:54.785750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afc2be-4117-4dd0-9f0d-50ba1ce17e8b None admin] RBAC: Adding query filter params (interface=public, service_id=4ee9717008d842ecb8215cba2630e9c7, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:54.786156 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afc2be-4117-4dd0-9f0d-50ba1ce17e8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:54.787560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afc2be-4117-4dd0-9f0d-50ba1ce17e8b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:54.799471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 385/773] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 06:17:54 2022] GET /identity/v3/endpoints?service_id=4ee9717008d842ecb8215cba2630e9c7&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:54.800684 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afc2be-4117-4dd0-9f0d-50ba1ce17e8b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:54.806798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e6f29-1d2f-4068-a66f-49ccb1d84014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:54.807666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e6f29-1d2f-4068-a66f-49ccb1d84014 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:54.808096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e6f29-1d2f-4068-a66f-49ccb1d84014 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:54.808574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e6f29-1d2f-4068-a66f-49ccb1d84014 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:54.809723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e6f29-1d2f-4068-a66f-49ccb1d84014 None admin] RBAC: Adding query filter params (interface=public, service_id=cdbd1551d4ce405a8cad5c9394c98b98, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:54.810208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e6f29-1d2f-4068-a66f-49ccb1d84014 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:54.811773 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e6f29-1d2f-4068-a66f-49ccb1d84014 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:54.822126 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 389/774] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 06:17:54 2022] GET /identity/v3/endpoints?service_id=cdbd1551d4ce405a8cad5c9394c98b98&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:54.823267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e6f29-1d2f-4068-a66f-49ccb1d84014 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:54.922205 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afc2be-4117-4dd0-9f0d-50ba1ce17e8b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:54.936573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e6f29-1d2f-4068-a66f-49ccb1d84014 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:54.938928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:54.960454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d9f2a-dda0-48e3-8977-78aa23e224c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:54.961537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d9f2a-dda0-48e3-8977-78aa23e224c3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:54.962096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d9f2a-dda0-48e3-8977-78aa23e224c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:54.962702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d9f2a-dda0-48e3-8977-78aa23e224c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:54.963733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d9f2a-dda0-48e3-8977-78aa23e224c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:54.964244 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d9f2a-dda0-48e3-8977-78aa23e224c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:54.966674 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d9f2a-dda0-48e3-8977-78aa23e224c3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:54.975338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38e524-bfd3-4c24-b882-e34400889204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:54.976217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38e524-bfd3-4c24-b882-e34400889204 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:54.976398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38e524-bfd3-4c24-b882-e34400889204 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:54.976671 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38e524-bfd3-4c24-b882-e34400889204 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:54.977366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38e524-bfd3-4c24-b882-e34400889204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:54.978941 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38e524-bfd3-4c24-b882-e34400889204 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:54.982758 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 386/775] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:54.984021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d9f2a-dda0-48e3-8977-78aa23e224c3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:54.989770 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 390/776] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 06:17:54 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:54.991798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38e524-bfd3-4c24-b882-e34400889204 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:55.018130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:55.158964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d9f2a-dda0-48e3-8977-78aa23e224c3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:55.207962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38e524-bfd3-4c24-b882-e34400889204 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:55.208142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999d713-cb92-437d-aed3-524d6ded1b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:55.208684 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999d713-cb92-437d-aed3-524d6ded1b4f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:55.208969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999d713-cb92-437d-aed3-524d6ded1b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:55.209251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999d713-cb92-437d-aed3-524d6ded1b4f None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:55.210012 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999d713-cb92-437d-aed3-524d6ded1b4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:55.211368 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999d713-cb92-437d-aed3-524d6ded1b4f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:55.223720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 387/777] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 06:17:54 2022] GET /identity/v3/services => generated 772 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:55.253001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14694f3d-c328-45cc-a9cd-07af0d97432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:55.254205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14694f3d-c328-45cc-a9cd-07af0d97432f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:55.254640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14694f3d-c328-45cc-a9cd-07af0d97432f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:55.255063 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14694f3d-c328-45cc-a9cd-07af0d97432f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:55.264072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14694f3d-c328-45cc-a9cd-07af0d97432f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:55.265847 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14694f3d-c328-45cc-a9cd-07af0d97432f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-14694f3d-c328-45cc-a9cd-07af0d97432f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:55.281597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:55.282760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:55.283876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:55.283876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:55.283876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:55.283876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:55.283876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:55.283876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:55.283876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:55.283876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:55.286273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 391/778] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:55.292258 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999d713-cb92-437d-aed3-524d6ded1b4f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:55.360864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823479d8-2670-4c2f-b513-3d99cb20a88d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:55.361529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823479d8-2670-4c2f-b513-3d99cb20a88d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:55.362059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823479d8-2670-4c2f-b513-3d99cb20a88d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:55.363906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 392/779] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:55.370232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b6760-60d1-4d89-a1ff-932418ab32b8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:55.370549 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b6760-60d1-4d89-a1ff-932418ab32b8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:55.370788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b6760-60d1-4d89-a1ff-932418ab32b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:55.405381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7e2b6760-60d1-4d89-a1ff-932418ab32b8 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:55.406013 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7e2b6760-60d1-4d89-a1ff-932418ab32b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:55.439211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:55.446316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:55.463440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999d713-cb92-437d-aed3-524d6ded1b4f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:55.498526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 393/780] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:17:55.509145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a798f-f6a6-43fb-ab36-a69c130b751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:55.509817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a798f-f6a6-43fb-ab36-a69c130b751f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:55.510131 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a798f-f6a6-43fb-ab36-a69c130b751f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:55.510410 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a798f-f6a6-43fb-ab36-a69c130b751f None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:55.511202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a798f-f6a6-43fb-ab36-a69c130b751f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:55.511512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a798f-f6a6-43fb-ab36-a69c130b751f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:55.512987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a798f-f6a6-43fb-ab36-a69c130b751f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:55.521052 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b6760-60d1-4d89-a1ff-932418ab32b8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:55.526338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 388/781] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:55.538034 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a798f-f6a6-43fb-ab36-a69c130b751f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:55.699372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a798f-f6a6-43fb-ab36-a69c130b751f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:55.741572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa25c6e-c3e8-4c52-95ec-3045f2b145ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:55.742575 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa25c6e-c3e8-4c52-95ec-3045f2b145ee None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:55.744624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa25c6e-c3e8-4c52-95ec-3045f2b145ee None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:55.744910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa25c6e-c3e8-4c52-95ec-3045f2b145ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:55.745760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa25c6e-c3e8-4c52-95ec-3045f2b145ee None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=e0ff633636ec403780dd4bd21e0fa65a) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:55.746103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa25c6e-c3e8-4c52-95ec-3045f2b145ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:55.749333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa25c6e-c3e8-4c52-95ec-3045f2b145ee None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:55.756711 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b6760-60d1-4d89-a1ff-932418ab32b8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:55.769614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 389/782] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:55 2022] GET /identity/v3/role_assignments?user.id=e0ff633636ec403780dd4bd21e0fa65a&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:55.801179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711aec3-3b75-4372-95f9-ccdb22447122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:55.802047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711aec3-3b75-4372-95f9-ccdb22447122 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:55.802400 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711aec3-3b75-4372-95f9-ccdb22447122 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:55.802677 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711aec3-3b75-4372-95f9-ccdb22447122 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:55.815749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711aec3-3b75-4372-95f9-ccdb22447122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a711aec3-3b75-4372-95f9-ccdb22447122 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:55.823783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:55.826776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:55.828740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 394/783] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:55.835312 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa25c6e-c3e8-4c52-95ec-3045f2b145ee None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:55.853817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:55.961967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa25c6e-c3e8-4c52-95ec-3045f2b145ee None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:56.001706 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad18479-299c-4187-b48c-0dc8b8851d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:56.002307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18479-299c-4187-b48c-0dc8b8851d06 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:56.002590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18479-299c-4187-b48c-0dc8b8851d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:56.002863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18479-299c-4187-b48c-0dc8b8851d06 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:56.003608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad18479-299c-4187-b48c-0dc8b8851d06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:56.003910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad18479-299c-4187-b48c-0dc8b8851d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:56.005274 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad18479-299c-4187-b48c-0dc8b8851d06 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:56.015393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 390/784] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:56.021876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711aec3-3b75-4372-95f9-ccdb22447122 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:56.067341 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:56.186008 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711aec3-3b75-4372-95f9-ccdb22447122 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:56.236061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abe607-1b09-49eb-9d16-c1942f936b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:56.236865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe607-1b09-49eb-9d16-c1942f936b32 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:56.237145 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe607-1b09-49eb-9d16-c1942f936b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:56.237420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe607-1b09-49eb-9d16-c1942f936b32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:56.249816 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abe607-1b09-49eb-9d16-c1942f936b32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:56.251576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abe607-1b09-49eb-9d16-c1942f936b32 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c2abe607-1b09-49eb-9d16-c1942f936b32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:56.275205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:56.276142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:56.277048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:56.277048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:56.277048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:56.277048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:56.280018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 395/785] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:56.285299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad18479-299c-4187-b48c-0dc8b8851d06 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:56.398189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:56.439204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad18479-299c-4187-b48c-0dc8b8851d06 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:56.489379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc863e91-1b17-40a6-97da-b9ad9ea239a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:56.490079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc863e91-1b17-40a6-97da-b9ad9ea239a4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:56.490364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc863e91-1b17-40a6-97da-b9ad9ea239a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:56.490639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc863e91-1b17-40a6-97da-b9ad9ea239a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:56.491365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc863e91-1b17-40a6-97da-b9ad9ea239a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:56.491674 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc863e91-1b17-40a6-97da-b9ad9ea239a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:56.493041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc863e91-1b17-40a6-97da-b9ad9ea239a4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:56.510348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 391/786] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:56.516332 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abe607-1b09-49eb-9d16-c1942f936b32 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:56.662752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:56.706110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abe607-1b09-49eb-9d16-c1942f936b32 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:56.755051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bf2a8-ef3a-4106-a5a5-4f2b6066b720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:56.755861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bf2a8-ef3a-4106-a5a5-4f2b6066b720 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:56.756149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bf2a8-ef3a-4106-a5a5-4f2b6066b720 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:56.756431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bf2a8-ef3a-4106-a5a5-4f2b6066b720 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:56.770143 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bf2a8-ef3a-4106-a5a5-4f2b6066b720 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:56.771629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bf2a8-ef3a-4106-a5a5-4f2b6066b720 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5d3bf2a8-ef3a-4106-a5a5-4f2b6066b720 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:56.792895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:56.794569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:56.796132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:56.796132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:56.796132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:56.796132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:56.796132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:56.796132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:56.796132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:56.796132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:56.796132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:17:56.796132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:56.796132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 396/787] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 06:17:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:56.800326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc863e91-1b17-40a6-97da-b9ad9ea239a4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:56.943470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc863e91-1b17-40a6-97da-b9ad9ea239a4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:56.978952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:56.987544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899441f7-576d-485c-bab2-eac7a064cf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:56.988235 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899441f7-576d-485c-bab2-eac7a064cf97 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:56.988518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899441f7-576d-485c-bab2-eac7a064cf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:56.988793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899441f7-576d-485c-bab2-eac7a064cf97 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:56.989634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899441f7-576d-485c-bab2-eac7a064cf97 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:56.990017 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899441f7-576d-485c-bab2-eac7a064cf97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:56.991448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899441f7-576d-485c-bab2-eac7a064cf97 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:57.015294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 392/788] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 06:17:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:17:57.020673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bf2a8-ef3a-4106-a5a5-4f2b6066b720 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:57.131686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373ad98-b7b0-4485-85a0-561330c989d3 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:57.132002 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373ad98-b7b0-4485-85a0-561330c989d3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:57.132262 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373ad98-b7b0-4485-85a0-561330c989d3 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:57.133849 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 393/789] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:17:57.139758 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f407cc-3ba9-4c78-b3db-49c984cbc2af None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:57.140042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f407cc-3ba9-4c78-b3db-49c984cbc2af None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:57.140323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f407cc-3ba9-4c78-b3db-49c984cbc2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:57.158305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-01f407cc-3ba9-4c78-b3db-49c984cbc2af None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:57.159511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-01f407cc-3ba9-4c78-b3db-49c984cbc2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:57.214527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bf2a8-ef3a-4106-a5a5-4f2b6066b720 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:57.257097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 394/790] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:17:57.268400 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:57.279322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35083c3-c52e-4ca1-80e5-0acc200613cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:57.279998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35083c3-c52e-4ca1-80e5-0acc200613cb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:57.280287 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35083c3-c52e-4ca1-80e5-0acc200613cb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:57.280574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35083c3-c52e-4ca1-80e5-0acc200613cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:57.328510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f407cc-3ba9-4c78-b3db-49c984cbc2af None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:57.360513 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35083c3-c52e-4ca1-80e5-0acc200613cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:57.362007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35083c3-c52e-4ca1-80e5-0acc200613cb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:57.464329 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 397/791] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:17:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:17:57.471676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35083c3-c52e-4ca1-80e5-0acc200613cb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:57.530344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:57.553451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f407cc-3ba9-4c78-b3db-49c984cbc2af None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:57.583136 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:57.615914 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c8647-749a-404d-8b4f-fc327bd2b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:57.617110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c8647-749a-404d-8b4f-fc327bd2b513 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:57.617774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c8647-749a-404d-8b4f-fc327bd2b513 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:57.618217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c8647-749a-404d-8b4f-fc327bd2b513 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:57.630574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c8647-749a-404d-8b4f-fc327bd2b513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1f0c8647-749a-404d-8b4f-fc327bd2b513 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:57.639548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:57.640484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:57.641309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 395/792] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:17:57.648946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9c995-fc97-4b92-9793-d0943d36fc2a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:57.648946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9c995-fc97-4b92-9793-d0943d36fc2a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:57.648946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9c995-fc97-4b92-9793-d0943d36fc2a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:57.648946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 396/793] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:57.648946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9c995-fc97-4b92-9793-d0943d36fc2a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:57.653650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35083c3-c52e-4ca1-80e5-0acc200613cb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:57.692698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530cc6b-4dcc-45df-88e5-2900b7b597fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:57.693844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530cc6b-4dcc-45df-88e5-2900b7b597fc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:57.694397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530cc6b-4dcc-45df-88e5-2900b7b597fc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:57.694871 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530cc6b-4dcc-45df-88e5-2900b7b597fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:57.710469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530cc6b-4dcc-45df-88e5-2900b7b597fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:57.712281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530cc6b-4dcc-45df-88e5-2900b7b597fc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f530cc6b-4dcc-45df-88e5-2900b7b597fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:57.735425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:57.736463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:57.737339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:57.737339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:57.737339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:57.737339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:57.739653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 398/794] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:57.741900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588525b9-33d7-4fec-a3fa-c0d9fae94b57 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:57.742436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588525b9-33d7-4fec-a3fa-c0d9fae94b57 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:57.742948 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588525b9-33d7-4fec-a3fa-c0d9fae94b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:57.765171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-588525b9-33d7-4fec-a3fa-c0d9fae94b57 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:57.766841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-588525b9-33d7-4fec-a3fa-c0d9fae94b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:57.774907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9c995-fc97-4b92-9793-d0943d36fc2a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:57.815422 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a63c14-0fe6-43dd-8473-e2b4d1a095e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:57.816089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a63c14-0fe6-43dd-8473-e2b4d1a095e7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:57.816170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a63c14-0fe6-43dd-8473-e2b4d1a095e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:57.816451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a63c14-0fe6-43dd-8473-e2b4d1a095e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:57.817284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a63c14-0fe6-43dd-8473-e2b4d1a095e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:57.817595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a63c14-0fe6-43dd-8473-e2b4d1a095e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:57.819096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a63c14-0fe6-43dd-8473-e2b4d1a095e7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:57.834412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 397/795] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:17:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:57.834412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a63c14-0fe6-43dd-8473-e2b4d1a095e7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:57.870193 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 399/796] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:17:57.872743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588525b9-33d7-4fec-a3fa-c0d9fae94b57 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:57.880973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:57.894653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:57.947454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a63c14-0fe6-43dd-8473-e2b4d1a095e7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:57.986499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88969acb-acca-40dc-9c2f-97d901d1782b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:57.987094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88969acb-acca-40dc-9c2f-97d901d1782b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:57.987382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88969acb-acca-40dc-9c2f-97d901d1782b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:57.987665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88969acb-acca-40dc-9c2f-97d901d1782b None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:57.988403 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88969acb-acca-40dc-9c2f-97d901d1782b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:57.988714 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88969acb-acca-40dc-9c2f-97d901d1782b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:57.990143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88969acb-acca-40dc-9c2f-97d901d1782b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:58.003237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 398/797] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:58.004546 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62822dfa-9e0a-4a8c-a97f-6064f020510f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.004842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62822dfa-9e0a-4a8c-a97f-6064f020510f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.005121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62822dfa-9e0a-4a8c-a97f-6064f020510f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.006519 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 399/798] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:58.010585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588525b9-33d7-4fec-a3fa-c0d9fae94b57 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:58.011193 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62822dfa-9e0a-4a8c-a97f-6064f020510f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:58.055925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cd4b5-85a2-410f-8fe4-2e914c9e883d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:58.056834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cd4b5-85a2-410f-8fe4-2e914c9e883d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.057170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cd4b5-85a2-410f-8fe4-2e914c9e883d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.057453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cd4b5-85a2-410f-8fe4-2e914c9e883d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.070565 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cd4b5-85a2-410f-8fe4-2e914c9e883d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:58.072446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cd4b5-85a2-410f-8fe4-2e914c9e883d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-066cd4b5-85a2-410f-8fe4-2e914c9e883d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:58.104840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:58.105820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:58.106727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:58.106727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:58.106727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:58.106727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:58.106727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 400/799] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:58.107007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cd4b5-85a2-410f-8fe4-2e914c9e883d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:58.156745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62822dfa-9e0a-4a8c-a97f-6064f020510f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:58.158408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:58.177869 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:58.203333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ee186-d760-4a88-8845-b8fc52c33700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:58.204140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ee186-d760-4a88-8845-b8fc52c33700 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.204424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ee186-d760-4a88-8845-b8fc52c33700 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.204702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ee186-d760-4a88-8845-b8fc52c33700 None admin] PATH_INFO: `/v3/services/network` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.205375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ee186-d760-4a88-8845-b8fc52c33700 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:58.206785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ee186-d760-4a88-8845-b8fc52c33700 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3e2ee186-d760-4a88-8845-b8fc52c33700 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:58.220089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:58.221329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:58.222338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 400/800] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:58 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:58.223334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf6746-e508-4f89-97ee-778a24903794 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.223334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf6746-e508-4f89-97ee-778a24903794 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.223334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf6746-e508-4f89-97ee-778a24903794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.231513 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cd4b5-85a2-410f-8fe4-2e914c9e883d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:58.241455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-19bf6746-e508-4f89-97ee-778a24903794 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:58.242594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-19bf6746-e508-4f89-97ee-778a24903794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:58.271495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309ff40-5e83-4c53-9939-85ef12077cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:58.272286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309ff40-5e83-4c53-9939-85ef12077cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.272823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309ff40-5e83-4c53-9939-85ef12077cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.273104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309ff40-5e83-4c53-9939-85ef12077cb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.282215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309ff40-5e83-4c53-9939-85ef12077cb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:58.283779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309ff40-5e83-4c53-9939-85ef12077cb0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c309ff40-5e83-4c53-9939-85ef12077cb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:17:58.299891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:58.300846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:58.301751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:17:58.301751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:17:58.301751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:17:58.301751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:17:58.301751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:17:58.301751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:17:58.301751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:17:58.301751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:58.301751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 401/801] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:58.302266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309ff40-5e83-4c53-9939-85ef12077cb0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:58.320591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 401/802] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:17:58.322636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598da57-bb31-45a1-9f14-3eced32e71b4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.322930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598da57-bb31-45a1-9f14-3eced32e71b4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.323311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598da57-bb31-45a1-9f14-3eced32e71b4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.324967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 402/803] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:17:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:17:58.325677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598da57-bb31-45a1-9f14-3eced32e71b4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:58.422494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309ff40-5e83-4c53-9939-85ef12077cb0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:58.443028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:58.445245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:58.451119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598da57-bb31-45a1-9f14-3eced32e71b4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:58.456548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af11e4-7104-4a3a-b068-e43d2d4a336f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:58.457210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af11e4-7104-4a3a-b068-e43d2d4a336f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.457492 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af11e4-7104-4a3a-b068-e43d2d4a336f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.457770 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af11e4-7104-4a3a-b068-e43d2d4a336f None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.458525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af11e4-7104-4a3a-b068-e43d2d4a336f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:58.458840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af11e4-7104-4a3a-b068-e43d2d4a336f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:58.460191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af11e4-7104-4a3a-b068-e43d2d4a336f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:58.473148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 402/804] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:58.474196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af11e4-7104-4a3a-b068-e43d2d4a336f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:58.487095 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22186daf-3b6f-4607-b7cb-cc1ddf358f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:58.487765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186daf-3b6f-4607-b7cb-cc1ddf358f30 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.488053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186daf-3b6f-4607-b7cb-cc1ddf358f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.488338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186daf-3b6f-4607-b7cb-cc1ddf358f30 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.489057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22186daf-3b6f-4607-b7cb-cc1ddf358f30 None admin] RBAC: Adding query filter params (name=network) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:58.489372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22186daf-3b6f-4607-b7cb-cc1ddf358f30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:58.490778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22186daf-3b6f-4607-b7cb-cc1ddf358f30 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:58.495380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 403/805] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:17:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:58.496790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b1b9e-70c5-46c9-9f87-27015a657490 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.497081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b1b9e-70c5-46c9-9f87-27015a657490 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.497364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b1b9e-70c5-46c9-9f87-27015a657490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.513620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8d9b1b9e-70c5-46c9-9f87-27015a657490 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:17:58.514835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8d9b1b9e-70c5-46c9-9f87-27015a657490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:17:58.572084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af11e4-7104-4a3a-b068-e43d2d4a336f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:58.581242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 404/806] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:17:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:17:58.582126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b1b9e-70c5-46c9-9f87-27015a657490 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:58.607587 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598d3c5-dfc8-4733-9ef0-420d556570a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:58.608264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598d3c5-dfc8-4733-9ef0-420d556570a6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.608559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598d3c5-dfc8-4733-9ef0-420d556570a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.608842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598d3c5-dfc8-4733-9ef0-420d556570a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.609644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598d3c5-dfc8-4733-9ef0-420d556570a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:58.609984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598d3c5-dfc8-4733-9ef0-420d556570a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:58.611416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598d3c5-dfc8-4733-9ef0-420d556570a6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:58.622877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 403/807] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:17:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:58.623681 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598d3c5-dfc8-4733-9ef0-420d556570a6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:58.673734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:58.677433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:58.687197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b1b9e-70c5-46c9-9f87-27015a657490 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:58.722275 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8182e-17aa-4812-b7c2-f91786001b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:58.723082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8182e-17aa-4812-b7c2-f91786001b17 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.723363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8182e-17aa-4812-b7c2-f91786001b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.723641 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8182e-17aa-4812-b7c2-f91786001b17 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.724299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8182e-17aa-4812-b7c2-f91786001b17 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:58.725671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8182e-17aa-4812-b7c2-f91786001b17 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:58.729358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598d3c5-dfc8-4733-9ef0-420d556570a6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-98a8182e-17aa-4812-b7c2-f91786001b17 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:58.737647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:58.738731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:58.739557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 405/808] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Jul 21 06:17:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:58.740405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8182e-17aa-4812-b7c2-f91786001b17 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:58.768450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4960e8-9c5d-4a75-be01-744184094e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:58.769190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4960e8-9c5d-4a75-be01-744184094e99 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.769477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4960e8-9c5d-4a75-be01-744184094e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.769762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4960e8-9c5d-4a75-be01-744184094e99 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.778233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4960e8-9c5d-4a75-be01-744184094e99 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:58.779779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4960e8-9c5d-4a75-be01-744184094e99 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4c4960e8-9c5d-4a75-be01-744184094e99 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:17:58.794640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:58.795812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 404/809] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:17:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:58.797077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4960e8-9c5d-4a75-be01-744184094e99 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:58.844443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8182e-17aa-4812-b7c2-f91786001b17 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:58.878264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16b600-3958-4f50-87a1-700fb3f8a4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:58.879279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16b600-3958-4f50-87a1-700fb3f8a4ed None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.879691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16b600-3958-4f50-87a1-700fb3f8a4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.879978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16b600-3958-4f50-87a1-700fb3f8a4ed None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.880689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16b600-3958-4f50-87a1-700fb3f8a4ed None admin] RBAC: Adding query filter params (type=network) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:58.880995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16b600-3958-4f50-87a1-700fb3f8a4ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:58.882395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16b600-3958-4f50-87a1-700fb3f8a4ed None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:58.887863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 406/810] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:17:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:58.888721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16b600-3958-4f50-87a1-700fb3f8a4ed None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:58.897764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4960e8-9c5d-4a75-be01-744184094e99 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:58.900102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:58.909765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:58.931964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c097a41-0ba2-405e-b83a-a551cab33f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:58.932752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c097a41-0ba2-405e-b83a-a551cab33f6f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:58.933031 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c097a41-0ba2-405e-b83a-a551cab33f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:58.933305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c097a41-0ba2-405e-b83a-a551cab33f6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:58.944134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c097a41-0ba2-405e-b83a-a551cab33f6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4c097a41-0ba2-405e-b83a-a551cab33f6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:17:58.952658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:58.953829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 405/811] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:17:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:58.954857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c097a41-0ba2-405e-b83a-a551cab33f6f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:58.993653 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16b600-3958-4f50-87a1-700fb3f8a4ed None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:59.028886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cc670-8ee0-416f-bedb-84867648e841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:59.029474 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cc670-8ee0-416f-bedb-84867648e841 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:59.029763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cc670-8ee0-416f-bedb-84867648e841 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:59.030077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cc670-8ee0-416f-bedb-84867648e841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:59.030917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62cc670-8ee0-416f-bedb-84867648e841 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=058f099e6f7545a1b2734dbe5afd2027) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:59.031228 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62cc670-8ee0-416f-bedb-84867648e841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:59.032669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62cc670-8ee0-416f-bedb-84867648e841 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:59.038657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 407/812] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:17:58 2022] GET /identity/v3/role_assignments?user.id=058f099e6f7545a1b2734dbe5afd2027&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:59.039440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cc670-8ee0-416f-bedb-84867648e841 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:59.066155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c097a41-0ba2-405e-b83a-a551cab33f6f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:59.107100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c137e-546e-4c86-a1ea-6d99972a131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:59.107787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c137e-546e-4c86-a1ea-6d99972a131f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:59.108073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c137e-546e-4c86-a1ea-6d99972a131f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:59.108353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c137e-546e-4c86-a1ea-6d99972a131f None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:59.109088 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c137e-546e-4c86-a1ea-6d99972a131f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:59.109399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c137e-546e-4c86-a1ea-6d99972a131f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:59.110836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c137e-546e-4c86-a1ea-6d99972a131f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:59.120462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 406/813] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 06:17:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:59.120462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c137e-546e-4c86-a1ea-6d99972a131f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:59.146163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:59.151607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:59.168384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cc670-8ee0-416f-bedb-84867648e841 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:59.207391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1e809-8bce-4cc4-9ab9-2365554818d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:59.208072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e809-8bce-4cc4-9ab9-2365554818d4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:59.208357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e809-8bce-4cc4-9ab9-2365554818d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:59.208637 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e809-8bce-4cc4-9ab9-2365554818d4 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:59.209490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1e809-8bce-4cc4-9ab9-2365554818d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:59.209803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1e809-8bce-4cc4-9ab9-2365554818d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:59.211321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1e809-8bce-4cc4-9ab9-2365554818d4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:59.231179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 408/814] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 06:17:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:59.232667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1e809-8bce-4cc4-9ab9-2365554818d4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:59.245916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c137e-546e-4c86-a1ea-6d99972a131f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:59.288194 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9b56c-f57d-4cb2-8635-27601cefab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:59.288787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b56c-f57d-4cb2-8635-27601cefab96 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:59.289058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b56c-f57d-4cb2-8635-27601cefab96 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:59.289333 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b56c-f57d-4cb2-8635-27601cefab96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:59.290272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9b56c-f57d-4cb2-8635-27601cefab96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:59.291459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9b56c-f57d-4cb2-8635-27601cefab96 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:59.315866 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 407/815] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 06:17:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:17:59.316699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9b56c-f57d-4cb2-8635-27601cefab96 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:59.353395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1e809-8bce-4cc4-9ab9-2365554818d4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:59.426155 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f6e3a-a8fb-427e-9ae3-4d3a1c246ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:59.426897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f6e3a-a8fb-427e-9ae3-4d3a1c246ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:59.427184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f6e3a-a8fb-427e-9ae3-4d3a1c246ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:59.427467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f6e3a-a8fb-427e-9ae3-4d3a1c246ec8 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:59.428537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f6e3a-a8fb-427e-9ae3-4d3a1c246ec8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:59.428891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f6e3a-a8fb-427e-9ae3-4d3a1c246ec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:59.431229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f6e3a-a8fb-427e-9ae3-4d3a1c246ec8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:59.433100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:59.436337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:59.446162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 409/816] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:17:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:59.447219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f6e3a-a8fb-427e-9ae3-4d3a1c246ec8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:59.457577 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9b56c-f57d-4cb2-8635-27601cefab96 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:59.502270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2856e1-cfc5-4e2d-950d-6d967de4a438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:59.502951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2856e1-cfc5-4e2d-950d-6d967de4a438 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:59.503233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2856e1-cfc5-4e2d-950d-6d967de4a438 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:59.503509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2856e1-cfc5-4e2d-950d-6d967de4a438 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:59.504229 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2856e1-cfc5-4e2d-950d-6d967de4a438 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:59.504542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2856e1-cfc5-4e2d-950d-6d967de4a438 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:59.505904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2856e1-cfc5-4e2d-950d-6d967de4a438 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:59.516042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 408/817] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 06:17:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:59.516042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2856e1-cfc5-4e2d-950d-6d967de4a438 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:59.583508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f6e3a-a8fb-427e-9ae3-4d3a1c246ec8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:59.622531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec5145-f39b-4ca8-a566-c7b251843a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:59.623346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec5145-f39b-4ca8-a566-c7b251843a3c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:59.623624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec5145-f39b-4ca8-a566-c7b251843a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:59.623993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec5145-f39b-4ca8-a566-c7b251843a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:59.635001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2856e1-cfc5-4e2d-950d-6d967de4a438 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:59.635212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec5145-f39b-4ca8-a566-c7b251843a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:59.636954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec5145-f39b-4ca8-a566-c7b251843a3c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d9ec5145-f39b-4ca8-a566-c7b251843a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:59.661106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:59.662309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:59.663226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:59.663226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:59.663226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:59.663226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:17:59.663447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 410/818] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:59.664146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec5145-f39b-4ca8-a566-c7b251843a3c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:59.673527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e003a-f23d-4c9d-bb88-df0b03662314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:59.674379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e003a-f23d-4c9d-bb88-df0b03662314 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:59.675072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e003a-f23d-4c9d-bb88-df0b03662314 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:59.675365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e003a-f23d-4c9d-bb88-df0b03662314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:59.687222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e003a-f23d-4c9d-bb88-df0b03662314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:59.688667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e003a-f23d-4c9d-bb88-df0b03662314 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:59.697401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1b1e003a-f23d-4c9d-bb88-df0b03662314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:17:59.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:17:59.708668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:17:59.709543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:17:59.709543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:17:59.709543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:17:59.709543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:17:59.717102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 409/819] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:17:59.717962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e003a-f23d-4c9d-bb88-df0b03662314 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:59.722353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:17:59.779483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec5145-f39b-4ca8-a566-c7b251843a3c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:59.816824 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995030-7e20-4a9b-ab9c-58a211074b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:59.817554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995030-7e20-4a9b-ab9c-58a211074b24 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:59.817794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995030-7e20-4a9b-ab9c-58a211074b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:59.818105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995030-7e20-4a9b-ab9c-58a211074b24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:59.818979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e995030-7e20-4a9b-ab9c-58a211074b24 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:59.820202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e995030-7e20-4a9b-ab9c-58a211074b24 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:59.828090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e003a-f23d-4c9d-bb88-df0b03662314 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:17:59.850059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 411/820] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 06:17:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:17:59.850059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995030-7e20-4a9b-ab9c-58a211074b24 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:59.880111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d1d92-5d23-404e-a518-40a42808cfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:17:59.880913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d1d92-5d23-404e-a518-40a42808cfaf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:17:59.881277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d1d92-5d23-404e-a518-40a42808cfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:17:59.881630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d1d92-5d23-404e-a518-40a42808cfaf None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:17:59.882504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d1d92-5d23-404e-a518-40a42808cfaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:17:59.882889 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d1d92-5d23-404e-a518-40a42808cfaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:17:59.884452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d1d92-5d23-404e-a518-40a42808cfaf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:17:59.904352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 410/821] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:17:59.905970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d1d92-5d23-404e-a518-40a42808cfaf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:17:59.974294 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:00.001654 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995030-7e20-4a9b-ab9c-58a211074b24 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:00.012449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:00.036555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d1d92-5d23-404e-a518-40a42808cfaf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:00.051116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004102e2-8e02-46ce-bb61-f846995ce291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:00.051792 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004102e2-8e02-46ce-bb61-f846995ce291 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:00.052072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004102e2-8e02-46ce-bb61-f846995ce291 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:00.052347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004102e2-8e02-46ce-bb61-f846995ce291 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:00.053076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004102e2-8e02-46ce-bb61-f846995ce291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:00.053379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004102e2-8e02-46ce-bb61-f846995ce291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:00.054802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004102e2-8e02-46ce-bb61-f846995ce291 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:00.071766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 412/822] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:00.078076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004102e2-8e02-46ce-bb61-f846995ce291 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:00.091854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea668c-ad15-4c58-985d-3ffd718ae94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:00.093053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea668c-ad15-4c58-985d-3ffd718ae94c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:00.093394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea668c-ad15-4c58-985d-3ffd718ae94c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:00.093684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea668c-ad15-4c58-985d-3ffd718ae94c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:00.102780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea668c-ad15-4c58-985d-3ffd718ae94c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:00.104449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea668c-ad15-4c58-985d-3ffd718ae94c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-19ea668c-ad15-4c58-985d-3ffd718ae94c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:18:00.118936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:00.119839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:00.120709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:18:00.120709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:18:00.120709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:18:00.120709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:18:00.120709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:18:00.120709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:18:00.120709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:00.120709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:00.120709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 411/823] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:17:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:00.126749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea668c-ad15-4c58-985d-3ffd718ae94c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:00.219985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004102e2-8e02-46ce-bb61-f846995ce291 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:00.237273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea668c-ad15-4c58-985d-3ffd718ae94c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:00.255321 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:00.267112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca4c6d-b181-4d1c-806f-f54dd863f571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:00.269136 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4c6d-b181-4d1c-806f-f54dd863f571 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:00.269623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4c6d-b181-4d1c-806f-f54dd863f571 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:00.270231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4c6d-b181-4d1c-806f-f54dd863f571 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:00.276172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d920c6-4477-415d-96ce-b634928b80f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:00.276841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d920c6-4477-415d-96ce-b634928b80f5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:00.277182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d920c6-4477-415d-96ce-b634928b80f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:00.278054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d920c6-4477-415d-96ce-b634928b80f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:00.278868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d920c6-4477-415d-96ce-b634928b80f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:00.279184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d920c6-4477-415d-96ce-b634928b80f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:00.280617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d920c6-4477-415d-96ce-b634928b80f5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:00.281158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca4c6d-b181-4d1c-806f-f54dd863f571 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:00.282922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca4c6d-b181-4d1c-806f-f54dd863f571 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:00.293871 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 412/824] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:18:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:00.299222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d920c6-4477-415d-96ce-b634928b80f5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-50ca4c6d-b181-4d1c-806f-f54dd863f571 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:18:00.303645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:00.304561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:00.305505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:00.305505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:00.305505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:18:00.305505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:18:00.305505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:18:00.305505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:18:00.305505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:18:00.305505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:18:00.305505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:18:00.305505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:00.308221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 413/825] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 06:18:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:00.313979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca4c6d-b181-4d1c-806f-f54dd863f571 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:00.341650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:00.402418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d920c6-4477-415d-96ce-b634928b80f5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:00.418610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca4c6d-b181-4d1c-806f-f54dd863f571 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:00.438253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61e0d9-e198-4bb3-b24d-55bcb5d87d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:00.439148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61e0d9-e198-4bb3-b24d-55bcb5d87d28 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:00.439482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61e0d9-e198-4bb3-b24d-55bcb5d87d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:00.439862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61e0d9-e198-4bb3-b24d-55bcb5d87d28 None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/94c8eb61f2ca48288cba81d280a30747/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:00.441856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61e0d9-e198-4bb3-b24d-55bcb5d87d28 None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=94c8eb61f2ca48288cba81d280a30747, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:00.443514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61e0d9-e198-4bb3-b24d-55bcb5d87d28 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:00.455090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1cebc-e02d-4eab-82c8-05e4d79ecd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:00.455906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1cebc-e02d-4eab-82c8-05e4d79ecd65 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:00.456194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1cebc-e02d-4eab-82c8-05e4d79ecd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:00.456469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1cebc-e02d-4eab-82c8-05e4d79ecd65 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:00.457313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1cebc-e02d-4eab-82c8-05e4d79ecd65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:00.457703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1cebc-e02d-4eab-82c8-05e4d79ecd65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:00.458538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 413/826] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:18:00 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/94c8eb61f2ca48288cba81d280a30747/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:18:00.459406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1cebc-e02d-4eab-82c8-05e4d79ecd65 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:00.485405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 414/827] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 06:18:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:00.491156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61e0d9-e198-4bb3-b24d-55bcb5d87d28 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:00.532387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:00.605100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61e0d9-e198-4bb3-b24d-55bcb5d87d28 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:00.643360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b255a70-680d-4e0f-a475-27bd21bbde15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:00.644165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b255a70-680d-4e0f-a475-27bd21bbde15 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:00.644449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b255a70-680d-4e0f-a475-27bd21bbde15 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:00.644720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b255a70-680d-4e0f-a475-27bd21bbde15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:00.656517 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b255a70-680d-4e0f-a475-27bd21bbde15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:00.657896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b255a70-680d-4e0f-a475-27bd21bbde15 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6b255a70-680d-4e0f-a475-27bd21bbde15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:00.676547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:00.677441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:00.678413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:00.678413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:00.678413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:00.678413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:00.678413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 414/828] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:00.683423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1cebc-e02d-4eab-82c8-05e4d79ecd65 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:00.779096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:00.797028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1cebc-e02d-4eab-82c8-05e4d79ecd65 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:00.833700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22454f9-1833-4c02-8efc-654621b4cf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:00.834717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22454f9-1833-4c02-8efc-654621b4cf54 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:00.835178 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22454f9-1833-4c02-8efc-654621b4cf54 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:00.835650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22454f9-1833-4c02-8efc-654621b4cf54 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:00.836545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22454f9-1833-4c02-8efc-654621b4cf54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:00.837026 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22454f9-1833-4c02-8efc-654621b4cf54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:00.838593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22454f9-1833-4c02-8efc-654621b4cf54 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:00.851086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 415/829] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:00.857933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b255a70-680d-4e0f-a475-27bd21bbde15 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:00.952385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:00.970619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b255a70-680d-4e0f-a475-27bd21bbde15 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:01.007035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60660467-8418-49ad-8d6d-c8fc51e42da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:01.008182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60660467-8418-49ad-8d6d-c8fc51e42da6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:01.008670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60660467-8418-49ad-8d6d-c8fc51e42da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:01.009132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60660467-8418-49ad-8d6d-c8fc51e42da6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:01.017244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60660467-8418-49ad-8d6d-c8fc51e42da6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:01.018916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60660467-8418-49ad-8d6d-c8fc51e42da6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-60660467-8418-49ad-8d6d-c8fc51e42da6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:18:01.034471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:01.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:01.036350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:18:01.036350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:18:01.036350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:18:01.036350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:18:01.036350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:18:01.036350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:18:01.036350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:01.036350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:01.038471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 415/830] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:01.044204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22454f9-1833-4c02-8efc-654621b4cf54 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:01.155006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22454f9-1833-4c02-8efc-654621b4cf54 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:01.190317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac870cf2-7e80-4e4d-ae2f-0c445581167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:01.190983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac870cf2-7e80-4e4d-ae2f-0c445581167f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:01.191261 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac870cf2-7e80-4e4d-ae2f-0c445581167f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:01.191533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac870cf2-7e80-4e4d-ae2f-0c445581167f None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:01.192319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac870cf2-7e80-4e4d-ae2f-0c445581167f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:01.192625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac870cf2-7e80-4e4d-ae2f-0c445581167f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:01.194079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac870cf2-7e80-4e4d-ae2f-0c445581167f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:01.207760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 416/831] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:18:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:01.213367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60660467-8418-49ad-8d6d-c8fc51e42da6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:01.240102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04146f8e-e9f0-4d48-a00c-ac439d63189f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:01.240412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04146f8e-e9f0-4d48-a00c-ac439d63189f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:01.240654 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04146f8e-e9f0-4d48-a00c-ac439d63189f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:01.242152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 417/832] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:18:01.247764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659d039-d5c5-412f-bf98-5c38986c7d73 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:01.248053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659d039-d5c5-412f-bf98-5c38986c7d73 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:01.248333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659d039-d5c5-412f-bf98-5c38986c7d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:01.254235 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:01.265523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e659d039-d5c5-412f-bf98-5c38986c7d73 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:01.266741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e659d039-d5c5-412f-bf98-5c38986c7d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:01.335925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60660467-8418-49ad-8d6d-c8fc51e42da6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:01.346268 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 418/833] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:01.362186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659d039-d5c5-412f-bf98-5c38986c7d73 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:01.376269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306b6d6-9248-4365-8d79-e554547c630a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:01.377649 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306b6d6-9248-4365-8d79-e554547c630a None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:01.378131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306b6d6-9248-4365-8d79-e554547c630a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:01.378996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306b6d6-9248-4365-8d79-e554547c630a None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/e0ff633636ec403780dd4bd21e0fa65a/roles/7541347b464141a78014fd57dccc765b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:01.399434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306b6d6-9248-4365-8d79-e554547c630a None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=e0ff633636ec403780dd4bd21e0fa65a, role_id=7541347b464141a78014fd57dccc765b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:01.401925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306b6d6-9248-4365-8d79-e554547c630a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:01.414321 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:01.429965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 416/834] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:18:01 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/e0ff633636ec403780dd4bd21e0fa65a/roles/7541347b464141a78014fd57dccc765b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:18:01.485965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659d039-d5c5-412f-bf98-5c38986c7d73 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:01.526326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961046f1-b003-4d2e-be3f-62ab38435dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:01.527159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961046f1-b003-4d2e-be3f-62ab38435dad None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:01.528011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961046f1-b003-4d2e-be3f-62ab38435dad None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:01.528293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961046f1-b003-4d2e-be3f-62ab38435dad None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:01.529072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961046f1-b003-4d2e-be3f-62ab38435dad None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:01.530511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961046f1-b003-4d2e-be3f-62ab38435dad None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:01.532877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-961046f1-b003-4d2e-be3f-62ab38435dad None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:01.548333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:01.549340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 06:18:01.550208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:01.553401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 419/835] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Jul 21 06:18:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:01.559650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306b6d6-9248-4365-8d79-e554547c630a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:01.669140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306b6d6-9248-4365-8d79-e554547c630a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:01.707678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbd476-9e34-4f20-8b0d-68230860686a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:01.708710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbd476-9e34-4f20-8b0d-68230860686a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:01.709193 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbd476-9e34-4f20-8b0d-68230860686a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:01.709688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbd476-9e34-4f20-8b0d-68230860686a None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:01.710619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbd476-9e34-4f20-8b0d-68230860686a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:01.711125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbd476-9e34-4f20-8b0d-68230860686a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:01.712699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbd476-9e34-4f20-8b0d-68230860686a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:01.719030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 417/836] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 06:18:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:01.724868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961046f1-b003-4d2e-be3f-62ab38435dad None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:01.833602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961046f1-b003-4d2e-be3f-62ab38435dad None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:01.870889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24bc5c-b059-4d60-ab41-0537f5b4868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:01.872491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24bc5c-b059-4d60-ab41-0537f5b4868a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:01.872974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24bc5c-b059-4d60-ab41-0537f5b4868a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:01.873462 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24bc5c-b059-4d60-ab41-0537f5b4868a None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:01.874399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24bc5c-b059-4d60-ab41-0537f5b4868a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:01.874921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24bc5c-b059-4d60-ab41-0537f5b4868a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:01.876479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24bc5c-b059-4d60-ab41-0537f5b4868a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:01.881889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 420/837] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 06:18:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:02.036037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1ddc0-5bbc-45b0-ae41-8655094e0af1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:02.036366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1ddc0-5bbc-45b0-ae41-8655094e0af1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:02.036613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1ddc0-5bbc-45b0-ae41-8655094e0af1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:02.038303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 418/838] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:02.044196 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62be6d-9778-4d25-9bb3-956b9434a712 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:02.044486 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62be6d-9778-4d25-9bb3-956b9434a712 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:02.044773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62be6d-9778-4d25-9bb3-956b9434a712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:02.063853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1a62be6d-9778-4d25-9bb3-956b9434a712 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:02.065047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1a62be6d-9778-4d25-9bb3-956b9434a712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:02.123928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:02.142071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 421/839] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:02.158917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1ddc0-5bbc-45b0-ae41-8655094e0af1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:02.228168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:02.274181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1ddc0-5bbc-45b0-ae41-8655094e0af1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:02.311716 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9621e-6c9d-4076-ab81-095b0ad13628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:02.312865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9621e-6c9d-4076-ab81-095b0ad13628 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:02.313400 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9621e-6c9d-4076-ab81-095b0ad13628 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:02.313879 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9621e-6c9d-4076-ab81-095b0ad13628 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:02.314762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9621e-6c9d-4076-ab81-095b0ad13628 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:02.316369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9621e-6c9d-4076-ab81-095b0ad13628 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-cbb9621e-6c9d-4076-ab81-095b0ad13628 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:02.328722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:02.329722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 06:18:02.330591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:02.333383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 419/840] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:18:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:02.339217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62be6d-9778-4d25-9bb3-956b9434a712 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:02.446353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62be6d-9778-4d25-9bb3-956b9434a712 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:02.483065 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c94d28-4605-4182-a4cd-c37e15e48014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:02.484110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c94d28-4605-4182-a4cd-c37e15e48014 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:02.484600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c94d28-4605-4182-a4cd-c37e15e48014 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:02.485080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c94d28-4605-4182-a4cd-c37e15e48014 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:02.486116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c94d28-4605-4182-a4cd-c37e15e48014 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:02.486639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c94d28-4605-4182-a4cd-c37e15e48014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:02.488208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c94d28-4605-4182-a4cd-c37e15e48014 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:02.493561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 422/841] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:18:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:02.499874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9621e-6c9d-4076-ab81-095b0ad13628 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:02.610609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9621e-6c9d-4076-ab81-095b0ad13628 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:02.638442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:02.645890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4f816-d4f5-4e5a-b6bd-4ca5e357d755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:02.646656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4f816-d4f5-4e5a-b6bd-4ca5e357d755 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:02.646950 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4f816-d4f5-4e5a-b6bd-4ca5e357d755 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:02.647237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4f816-d4f5-4e5a-b6bd-4ca5e357d755 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:02.648226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4f816-d4f5-4e5a-b6bd-4ca5e357d755 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:02.648540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4f816-d4f5-4e5a-b6bd-4ca5e357d755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:02.649918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4f816-d4f5-4e5a-b6bd-4ca5e357d755 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:02.651423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45286b-56f7-4915-ab3d-ceea4fbfa8a1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:02.651715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45286b-56f7-4915-ab3d-ceea4fbfa8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:02.651996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45286b-56f7-4915-ab3d-ceea4fbfa8a1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:02.654461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 423/842] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:02.655873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 420/843] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:18:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:02.659540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0163d4d-dad9-4f6c-aa72-43d32038fe70 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:02.659830 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0163d4d-dad9-4f6c-aa72-43d32038fe70 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:02.660111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0163d4d-dad9-4f6c-aa72-43d32038fe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:02.680857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c0163d4d-dad9-4f6c-aa72-43d32038fe70 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:02.682181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c0163d4d-dad9-4f6c-aa72-43d32038fe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:02.867308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:02.886195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 424/844] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:02.910790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4f816-d4f5-4e5a-b6bd-4ca5e357d755 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:03.019477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4f816-d4f5-4e5a-b6bd-4ca5e357d755 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:03.067829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178b300-7dba-4c64-b31b-f0f463d5a0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:03.068646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178b300-7dba-4c64-b31b-f0f463d5a0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:03.068932 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178b300-7dba-4c64-b31b-f0f463d5a0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:03.069378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178b300-7dba-4c64-b31b-f0f463d5a0e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:03.081512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178b300-7dba-4c64-b31b-f0f463d5a0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a178b300-7dba-4c64-b31b-f0f463d5a0e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:03.089393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:03.090819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:03.091629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 421/845] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:18:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:03.098608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0163d4d-dad9-4f6c-aa72-43d32038fe70 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:03.201896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0163d4d-dad9-4f6c-aa72-43d32038fe70 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:03.236813 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bbb32-d153-454d-b7ab-5f4d59b01e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:03.237777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bbb32-d153-454d-b7ab-5f4d59b01e53 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:03.238289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bbb32-d153-454d-b7ab-5f4d59b01e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:03.238753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bbb32-d153-454d-b7ab-5f4d59b01e53 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:03.239669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bbb32-d153-454d-b7ab-5f4d59b01e53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:03.240150 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bbb32-d153-454d-b7ab-5f4d59b01e53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:03.241689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bbb32-d153-454d-b7ab-5f4d59b01e53 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:03.251507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 425/846] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:18:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:03.257246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178b300-7dba-4c64-b31b-f0f463d5a0e2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:03.310107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:03.365009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178b300-7dba-4c64-b31b-f0f463d5a0e2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:03.400077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11466c-2a7f-4329-b1f6-a352ac124a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:03.401128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11466c-2a7f-4329-b1f6-a352ac124a86 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:03.401591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11466c-2a7f-4329-b1f6-a352ac124a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:03.402112 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11466c-2a7f-4329-b1f6-a352ac124a86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:03.414187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11466c-2a7f-4329-b1f6-a352ac124a86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:03.415846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11466c-2a7f-4329-b1f6-a352ac124a86 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2c11466c-2a7f-4329-b1f6-a352ac124a86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:03.434545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:03.435650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:03.436738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:03.436738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:03.436738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:03.436738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:03.438603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 422/847] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:03.445973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bbb32-d153-454d-b7ab-5f4d59b01e53 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:03.503852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:03.552259 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bbb32-d153-454d-b7ab-5f4d59b01e53 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:03.589565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337474a-2f1f-4f8e-93cb-6cc68ad02952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:03.590311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337474a-2f1f-4f8e-93cb-6cc68ad02952 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:03.590598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337474a-2f1f-4f8e-93cb-6cc68ad02952 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:03.590880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337474a-2f1f-4f8e-93cb-6cc68ad02952 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:03.591608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337474a-2f1f-4f8e-93cb-6cc68ad02952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:03.591916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337474a-2f1f-4f8e-93cb-6cc68ad02952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:03.593289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337474a-2f1f-4f8e-93cb-6cc68ad02952 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:03.605827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 426/848] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:03.611181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11466c-2a7f-4329-b1f6-a352ac124a86 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:03.721110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11466c-2a7f-4329-b1f6-a352ac124a86 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:03.730858 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:03.733650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaa4a9-09cb-469b-8328-7835e111391c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:03.733958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaa4a9-09cb-469b-8328-7835e111391c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:03.734239 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaa4a9-09cb-469b-8328-7835e111391c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:03.736397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 427/849] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:03.743647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfb434-a36e-47db-9d8e-5308130173ce None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:03.743941 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfb434-a36e-47db-9d8e-5308130173ce None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:03.744220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfb434-a36e-47db-9d8e-5308130173ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:03.763602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-90dfb434-a36e-47db-9d8e-5308130173ce None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:03.764434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcb11b-ad28-4458-89dd-1187550e5369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:03.764711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-90dfb434-a36e-47db-9d8e-5308130173ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:03.765568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcb11b-ad28-4458-89dd-1187550e5369 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:03.766045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcb11b-ad28-4458-89dd-1187550e5369 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:03.766462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcb11b-ad28-4458-89dd-1187550e5369 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:03.775105 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bcb11b-ad28-4458-89dd-1187550e5369 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:03.776750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bcb11b-ad28-4458-89dd-1187550e5369 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e7bcb11b-ad28-4458-89dd-1187550e5369 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:18:03.791659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:03.792603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:03.794512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:03.794512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:03.794512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:18:03.794512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:18:03.794512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:18:03.794512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:18:03.794512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:18:03.794512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:18:03.794512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 06:18:03.794512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:03.797237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 423/850] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:18:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:03.802990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcb11b-ad28-4458-89dd-1187550e5369 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:03.846264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 428/851] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:03.865007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfb434-a36e-47db-9d8e-5308130173ce None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:03.914784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcb11b-ad28-4458-89dd-1187550e5369 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:03.930423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:03.953211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad8bb4-aa80-4719-bba4-f9f68b228481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:03.954249 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad8bb4-aa80-4719-bba4-f9f68b228481 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:03.954874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad8bb4-aa80-4719-bba4-f9f68b228481 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:03.955402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad8bb4-aa80-4719-bba4-f9f68b228481 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:03.956449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad8bb4-aa80-4719-bba4-f9f68b228481 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:03.956959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad8bb4-aa80-4719-bba4-f9f68b228481 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:03.958627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad8bb4-aa80-4719-bba4-f9f68b228481 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:03.971600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfb434-a36e-47db-9d8e-5308130173ce None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:03.982944 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 424/852] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 06:18:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:03.984137 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad8bb4-aa80-4719-bba4-f9f68b228481 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:04.008198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:04.008545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6732f-f07d-4ec6-b480-0ca942753cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:04.009357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6732f-f07d-4ec6-b480-0ca942753cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.009644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6732f-f07d-4ec6-b480-0ca942753cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.009961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6732f-f07d-4ec6-b480-0ca942753cd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.021798 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6732f-f07d-4ec6-b480-0ca942753cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9ce6732f-f07d-4ec6-b480-0ca942753cd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:04.030721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:04.031636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:04.032548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 429/853] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:18:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:04.033263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09baf54-9b4a-4af4-b4c8-901b4aba8ca5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.033828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09baf54-9b4a-4af4-b4c8-901b4aba8ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.033828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09baf54-9b4a-4af4-b4c8-901b4aba8ca5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.035905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 430/854] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:04.039028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09baf54-9b4a-4af4-b4c8-901b4aba8ca5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:04.107118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad8bb4-aa80-4719-bba4-f9f68b228481 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:04.144866 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c6238-7bc8-4546-874a-0c479ebd4adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:04.145523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c6238-7bc8-4546-874a-0c479ebd4adf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.145812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c6238-7bc8-4546-874a-0c479ebd4adf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.146144 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c6238-7bc8-4546-874a-0c479ebd4adf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.148210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:04.157079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09baf54-9b4a-4af4-b4c8-901b4aba8ca5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:04.196597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72187693-8fd8-4f5e-9b90-cc59dc878e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:04.197336 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72187693-8fd8-4f5e-9b90-cc59dc878e96 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.197550 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72187693-8fd8-4f5e-9b90-cc59dc878e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.197830 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72187693-8fd8-4f5e-9b90-cc59dc878e96 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.198795 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72187693-8fd8-4f5e-9b90-cc59dc878e96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:04.198916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72187693-8fd8-4f5e-9b90-cc59dc878e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:04.200866 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72187693-8fd8-4f5e-9b90-cc59dc878e96 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:04.213901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 431/855] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:18:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:04.213901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d29bb-9ff6-4617-8d28-899b7e78070e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.213901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d29bb-9ff6-4617-8d28-899b7e78070e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.213901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d29bb-9ff6-4617-8d28-899b7e78070e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.225331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c6238-7bc8-4546-874a-0c479ebd4adf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:04.227042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c6238-7bc8-4546-874a-0c479ebd4adf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:04.229354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-405d29bb-9ff6-4617-8d28-899b7e78070e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:04.230850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-405d29bb-9ff6-4617-8d28-899b7e78070e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:04.250712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:04.318366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 432/856] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:04.318366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d29bb-9ff6-4617-8d28-899b7e78070e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:04.342240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 425/857] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:03 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:18:04.343203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c6238-7bc8-4546-874a-0c479ebd4adf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:04.425432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:04.437836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d29bb-9ff6-4617-8d28-899b7e78070e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:04.453189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c6238-7bc8-4546-874a-0c479ebd4adf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:04.474580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:04.476246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a195c-dbb2-4576-bfa1-bcc49a1cb9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:04.477240 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a195c-dbb2-4576-bfa1-bcc49a1cb9cc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.477763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a195c-dbb2-4576-bfa1-bcc49a1cb9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.478240 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a195c-dbb2-4576-bfa1-bcc49a1cb9cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.489512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a195c-dbb2-4576-bfa1-bcc49a1cb9cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:04.490726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e20076-5480-41a0-932d-b5942868830f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:04.491409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e20076-5480-41a0-932d-b5942868830f None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.491544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a195c-dbb2-4576-bfa1-bcc49a1cb9cc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:04.491693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e20076-5480-41a0-932d-b5942868830f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.491975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e20076-5480-41a0-932d-b5942868830f None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.492702 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e20076-5480-41a0-932d-b5942868830f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:04.494253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e20076-5480-41a0-932d-b5942868830f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:04.510015 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 426/858] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:18:04 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:18:04.510816 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e20076-5480-41a0-932d-b5942868830f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fa6a195c-dbb2-4576-bfa1-bcc49a1cb9cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:04.511718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:04.512626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:04.513551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:04.513551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:04.513551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:04.513551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:04.517691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 433/859] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:04.518728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a195c-dbb2-4576-bfa1-bcc49a1cb9cc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:04.634677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a195c-dbb2-4576-bfa1-bcc49a1cb9cc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:04.673514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e20076-5480-41a0-932d-b5942868830f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:04.679734 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9b225-f611-4479-a608-15b3c1862f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:04.680412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9b225-f611-4479-a608-15b3c1862f4b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.680696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9b225-f611-4479-a608-15b3c1862f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.680977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9b225-f611-4479-a608-15b3c1862f4b None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.681714 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9b225-f611-4479-a608-15b3c1862f4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:04.682071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9b225-f611-4479-a608-15b3c1862f4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:04.684091 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9b225-f611-4479-a608-15b3c1862f4b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:04.691456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:04.698754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 434/860] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:04.704971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9b225-f611-4479-a608-15b3c1862f4b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:04.717343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603f8d9-22ea-4a25-a70d-b6cc6077fd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:04.718571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603f8d9-22ea-4a25-a70d-b6cc6077fd22 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.719066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603f8d9-22ea-4a25-a70d-b6cc6077fd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.719592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603f8d9-22ea-4a25-a70d-b6cc6077fd22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.732761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603f8d9-22ea-4a25-a70d-b6cc6077fd22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:04.735010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603f8d9-22ea-4a25-a70d-b6cc6077fd22 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2603f8d9-22ea-4a25-a70d-b6cc6077fd22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:04.755421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:04.756401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:04.757268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:04.757268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:04.757268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:04.757268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:04.757610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 427/861] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:04.764316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603f8d9-22ea-4a25-a70d-b6cc6077fd22 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:04.799990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:04.830949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9b225-f611-4479-a608-15b3c1862f4b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:04.874996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c8879-9cfa-4b38-bdcc-2d428b75ce0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:04.875946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8879-9cfa-4b38-bdcc-2d428b75ce0e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.876486 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8879-9cfa-4b38-bdcc-2d428b75ce0e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.876781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8879-9cfa-4b38-bdcc-2d428b75ce0e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.880017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603f8d9-22ea-4a25-a70d-b6cc6077fd22 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:04.888270 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8879-9cfa-4b38-bdcc-2d428b75ce0e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:04.890300 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8879-9cfa-4b38-bdcc-2d428b75ce0e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9d6c8879-9cfa-4b38-bdcc-2d428b75ce0e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:18:04.908551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:04.909537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:04.910506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:04.910506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:04.910506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:18:04.910506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:18:04.910506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:18:04.910506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:18:04.910506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:18:04.910506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:18:04.910506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:18:04.910506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:04.910506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 435/862] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 06:18:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:04.911906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c349-2a56-4f50-b0dc-ab920cc7afda None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.911906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c349-2a56-4f50-b0dc-ab920cc7afda None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.911906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c349-2a56-4f50-b0dc-ab920cc7afda None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.913618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 436/863] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:04.915118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2c349-2a56-4f50-b0dc-ab920cc7afda None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:04.923998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e293d-3cef-4fdc-aa1b-46bf336ed5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:04.924665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e293d-3cef-4fdc-aa1b-46bf336ed5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.924951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e293d-3cef-4fdc-aa1b-46bf336ed5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.925231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e293d-3cef-4fdc-aa1b-46bf336ed5c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.925990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e293d-3cef-4fdc-aa1b-46bf336ed5c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:04.926306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e293d-3cef-4fdc-aa1b-46bf336ed5c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:04.927668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e293d-3cef-4fdc-aa1b-46bf336ed5c9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:04.945730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 428/864] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:04.947101 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4e9a5-ce9f-4f6a-90e9-783f22d985f2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:04.947394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4e9a5-ce9f-4f6a-90e9-783f22d985f2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:04.947674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4e9a5-ce9f-4f6a-90e9-783f22d985f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:04.967406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-aaa4e9a5-ce9f-4f6a-90e9-783f22d985f2 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:04.968583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-aaa4e9a5-ce9f-4f6a-90e9-783f22d985f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:04.978712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:05.026585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2c349-2a56-4f50-b0dc-ab920cc7afda None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:05.038011 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 429/865] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:04 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:05.044329 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4e9a5-ce9f-4f6a-90e9-783f22d985f2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:05.068891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:05.070116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c5d4-3b66-44fb-9212-dbccaafdd963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:05.071077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c5d4-3b66-44fb-9212-dbccaafdd963 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:05.071633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c5d4-3b66-44fb-9212-dbccaafdd963 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:05.072091 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c5d4-3b66-44fb-9212-dbccaafdd963 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:05.073080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703c5d4-3b66-44fb-9212-dbccaafdd963 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:05.073584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703c5d4-3b66-44fb-9212-dbccaafdd963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:05.075773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703c5d4-3b66-44fb-9212-dbccaafdd963 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:05.099099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 437/866] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 06:18:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:05.099099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c5d4-3b66-44fb-9212-dbccaafdd963 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:05.166686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4e9a5-ce9f-4f6a-90e9-783f22d985f2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:05.202992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c5d4-3b66-44fb-9212-dbccaafdd963 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:05.205225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfdf28-366a-405e-8b8b-9be974cf4649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:05.206037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfdf28-366a-405e-8b8b-9be974cf4649 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:05.206322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfdf28-366a-405e-8b8b-9be974cf4649 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:05.206597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfdf28-366a-405e-8b8b-9be974cf4649 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:05.213340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfdf28-366a-405e-8b8b-9be974cf4649 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:05.215300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfdf28-366a-405e-8b8b-9be974cf4649 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:05.219672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7fdfdf28-366a-405e-8b8b-9be974cf4649 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:18:05.229125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:05.230066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:05.231515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:18:05.231515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:18:05.231515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:18:05.231515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:18:05.231515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:18:05.231515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:18:05.231515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:05.231515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:05.231515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 430/867] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:05.232651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfdf28-366a-405e-8b8b-9be974cf4649 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:05.247052 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c3d70-9cb1-43d9-b8d3-d98e5cfda79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:05.248013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c3d70-9cb1-43d9-b8d3-d98e5cfda79c None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:05.248303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c3d70-9cb1-43d9-b8d3-d98e5cfda79c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:05.248582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c3d70-9cb1-43d9-b8d3-d98e5cfda79c None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:05.249477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c3d70-9cb1-43d9-b8d3-d98e5cfda79c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:05.251257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c3d70-9cb1-43d9-b8d3-d98e5cfda79c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:05.267063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 438/868] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:18:05 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:18:05.267726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c3d70-9cb1-43d9-b8d3-d98e5cfda79c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:05.349344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:05.388460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfdf28-366a-405e-8b8b-9be974cf4649 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:05.397061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c3d70-9cb1-43d9-b8d3-d98e5cfda79c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:05.437480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388b5d2-0ca3-4cc4-ab19-b9185a8aac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:05.438768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388b5d2-0ca3-4cc4-ab19-b9185a8aac26 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:05.438768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388b5d2-0ca3-4cc4-ab19-b9185a8aac26 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:05.438968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b54f66-d803-47d8-876f-03fc82d38247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:05.439058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388b5d2-0ca3-4cc4-ab19-b9185a8aac26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:05.439257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b54f66-d803-47d8-876f-03fc82d38247 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:05.439476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b54f66-d803-47d8-876f-03fc82d38247 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:05.439758 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b54f66-d803-47d8-876f-03fc82d38247 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:05.440547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b54f66-d803-47d8-876f-03fc82d38247 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:05.440855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b54f66-d803-47d8-876f-03fc82d38247 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:05.443720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b54f66-d803-47d8-876f-03fc82d38247 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:05.458724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 439/869] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:18:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:05.466468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b54f66-d803-47d8-876f-03fc82d38247 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:05.505675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:05.523029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388b5d2-0ca3-4cc4-ab19-b9185a8aac26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:05.524766 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388b5d2-0ca3-4cc4-ab19-b9185a8aac26 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:05.598283 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b54f66-d803-47d8-876f-03fc82d38247 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:05.615969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:05.635293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a4f72-b28d-438d-a7cf-9fc8904e7191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:05.635949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a4f72-b28d-438d-a7cf-9fc8904e7191 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:05.636227 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a4f72-b28d-438d-a7cf-9fc8904e7191 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:05.636602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a4f72-b28d-438d-a7cf-9fc8904e7191 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:05.638039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a4f72-b28d-438d-a7cf-9fc8904e7191 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=94c8eb61f2ca48288cba81d280a30747) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:05.638358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a4f72-b28d-438d-a7cf-9fc8904e7191 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:05.639786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a4f72-b28d-438d-a7cf-9fc8904e7191 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:05.646448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 440/871] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:18:05 2022] GET /identity/v3/role_assignments?user.id=94c8eb61f2ca48288cba81d280a30747&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:05.650532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 431/871] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:05 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:18:05.655082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a4f72-b28d-438d-a7cf-9fc8904e7191 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:05.767905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:05.775118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a4f72-b28d-438d-a7cf-9fc8904e7191 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:05.815526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a4028-13a3-4eeb-829b-50b9c2e518c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:05.816327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a4028-13a3-4eeb-829b-50b9c2e518c3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:05.816604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a4028-13a3-4eeb-829b-50b9c2e518c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:05.816879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a4028-13a3-4eeb-829b-50b9c2e518c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:05.829633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a4028-13a3-4eeb-829b-50b9c2e518c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:05.831449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a4028-13a3-4eeb-829b-50b9c2e518c3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ca3a4028-13a3-4eeb-829b-50b9c2e518c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:05.850089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:05.851004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:05.851887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:05.851887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:05.851887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:05.851887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:05.852674 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 441/872] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:05.858800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388b5d2-0ca3-4cc4-ab19-b9185a8aac26 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:05.967986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388b5d2-0ca3-4cc4-ab19-b9185a8aac26 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:06.004494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3e80a-fe5d-494c-92d8-ef126d131033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:06.005558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3e80a-fe5d-494c-92d8-ef126d131033 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:06.006098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3e80a-fe5d-494c-92d8-ef126d131033 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:06.006583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3e80a-fe5d-494c-92d8-ef126d131033 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:06.007530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3e80a-fe5d-494c-92d8-ef126d131033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:06.008043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3e80a-fe5d-494c-92d8-ef126d131033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:06.009640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3e80a-fe5d-494c-92d8-ef126d131033 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:06.023604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 432/873] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:06.029404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a4028-13a3-4eeb-829b-50b9c2e518c3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:06.145357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a4028-13a3-4eeb-829b-50b9c2e518c3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:06.181120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7024410-db4b-4e1a-b5bb-699a6118f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:06.181930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7024410-db4b-4e1a-b5bb-699a6118f123 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:06.182237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7024410-db4b-4e1a-b5bb-699a6118f123 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:06.182517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7024410-db4b-4e1a-b5bb-699a6118f123 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:06.189618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7024410-db4b-4e1a-b5bb-699a6118f123 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:06.191112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7024410-db4b-4e1a-b5bb-699a6118f123 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c7024410-db4b-4e1a-b5bb-699a6118f123 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:18:06.206122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:06.207041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:06.207917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:18:06.207917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:18:06.207917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:18:06.207917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:18:06.207917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:18:06.207917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:18:06.207917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:06.207917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:06.207917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 442/874] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:06.215211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3e80a-fe5d-494c-92d8-ef126d131033 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:06.265977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:06.324508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3e80a-fe5d-494c-92d8-ef126d131033 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:06.359284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4b5da-2574-4db7-813e-b49ccb38b612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:06.359961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4b5da-2574-4db7-813e-b49ccb38b612 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:06.360250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4b5da-2574-4db7-813e-b49ccb38b612 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:06.360536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4b5da-2574-4db7-813e-b49ccb38b612 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:06.361337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4b5da-2574-4db7-813e-b49ccb38b612 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:06.361653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4b5da-2574-4db7-813e-b49ccb38b612 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:06.363205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4b5da-2574-4db7-813e-b49ccb38b612 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:06.376527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 433/875] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:18:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:06.382302 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7024410-db4b-4e1a-b5bb-699a6118f123 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:06.398323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:06.488458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7024410-db4b-4e1a-b5bb-699a6118f123 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:06.525273 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffbe94-0a35-49e5-8ec6-af23a701a24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:06.525957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffbe94-0a35-49e5-8ec6-af23a701a24d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:06.526248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffbe94-0a35-49e5-8ec6-af23a701a24d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:06.526531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffbe94-0a35-49e5-8ec6-af23a701a24d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:06.527800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffbe94-0a35-49e5-8ec6-af23a701a24d None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=e0ff633636ec403780dd4bd21e0fa65a) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:06.528007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffbe94-0a35-49e5-8ec6-af23a701a24d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:06.529420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffbe94-0a35-49e5-8ec6-af23a701a24d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:06.536036 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 443/876] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:18:06 2022] GET /identity/v3/role_assignments?user.id=e0ff633636ec403780dd4bd21e0fa65a&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:06.689284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f5b88-b555-4511-9e91-1f5154a28651 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:06.689512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f5b88-b555-4511-9e91-1f5154a28651 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:06.689795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f5b88-b555-4511-9e91-1f5154a28651 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:06.691205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 434/877] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:06.696636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33fb6-cde6-425b-a28b-86b69da13e03 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:06.697149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33fb6-cde6-425b-a28b-86b69da13e03 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:06.697636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33fb6-cde6-425b-a28b-86b69da13e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:06.715945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-47c33fb6-cde6-425b-a28b-86b69da13e03 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:06.717127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-47c33fb6-cde6-425b-a28b-86b69da13e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:06.790655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 444/878] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:06.814696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f5b88-b555-4511-9e91-1f5154a28651 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:06.916092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:06.921251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f5b88-b555-4511-9e91-1f5154a28651 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:06.959678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6afdd0-1c88-49b9-9079-54c58296315c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:06.960495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6afdd0-1c88-49b9-9079-54c58296315c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:06.960780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6afdd0-1c88-49b9-9079-54c58296315c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:06.961057 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6afdd0-1c88-49b9-9079-54c58296315c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:06.961780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6afdd0-1c88-49b9-9079-54c58296315c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:06.963182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6afdd0-1c88-49b9-9079-54c58296315c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0c6afdd0-1c88-49b9-9079-54c58296315c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:06.977012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:06.977974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:06.979383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 435/879] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Jul 21 06:18:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:06.984334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c33fb6-cde6-425b-a28b-86b69da13e03 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:06.986723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:07.095181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c33fb6-cde6-425b-a28b-86b69da13e03 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:07.130650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5bebd-8072-4d2d-8e33-cd5d8ea8d515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:07.131329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5bebd-8072-4d2d-8e33-cd5d8ea8d515 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:07.131615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5bebd-8072-4d2d-8e33-cd5d8ea8d515 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:07.131893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5bebd-8072-4d2d-8e33-cd5d8ea8d515 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:07.132615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5bebd-8072-4d2d-8e33-cd5d8ea8d515 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:07.132927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5bebd-8072-4d2d-8e33-cd5d8ea8d515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:07.134458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5bebd-8072-4d2d-8e33-cd5d8ea8d515 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:07.140358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 445/880] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 06:18:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:07.145620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6afdd0-1c88-49b9-9079-54c58296315c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:07.253553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6afdd0-1c88-49b9-9079-54c58296315c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:07.289014 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f82a4-9df7-4f4b-9dd8-a5f7e24388a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:07.289689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f82a4-9df7-4f4b-9dd8-a5f7e24388a5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:07.289998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f82a4-9df7-4f4b-9dd8-a5f7e24388a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:07.290285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f82a4-9df7-4f4b-9dd8-a5f7e24388a5 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:07.291007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f82a4-9df7-4f4b-9dd8-a5f7e24388a5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:07.291322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f82a4-9df7-4f4b-9dd8-a5f7e24388a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:07.292825 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f82a4-9df7-4f4b-9dd8-a5f7e24388a5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:07.302284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 436/881] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 06:18:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:07.304032 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5bebd-8072-4d2d-8e33-cd5d8ea8d515 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:07.376624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:07.413188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5bebd-8072-4d2d-8e33-cd5d8ea8d515 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:07.450293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e4b40-87b5-453b-8353-ae8996875a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:07.450963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e4b40-87b5-453b-8353-ae8996875a73 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:07.451250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e4b40-87b5-453b-8353-ae8996875a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:07.451531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e4b40-87b5-453b-8353-ae8996875a73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:07.452338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e4b40-87b5-453b-8353-ae8996875a73 None admin] RBAC: Adding query filter params (interface=public, service_id=2be9e84647aa4576a3a91cc4b5392775, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:07.452652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e4b40-87b5-453b-8353-ae8996875a73 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:07.454039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e4b40-87b5-453b-8353-ae8996875a73 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:07.459781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 446/882] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 06:18:07 2022] GET /identity/v3/endpoints?service_id=2be9e84647aa4576a3a91cc4b5392775&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:07.463623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b485a-06fa-4fc9-91a7-6e734c051e5e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:07.464219 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b485a-06fa-4fc9-91a7-6e734c051e5e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:07.464829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b485a-06fa-4fc9-91a7-6e734c051e5e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:07.465113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e4b40-87b5-453b-8353-ae8996875a73 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:07.470524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 437/883] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:07.472270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e675e-6bfb-4534-816c-d73da977fa9b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:07.472765 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e675e-6bfb-4534-816c-d73da977fa9b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:07.473258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e675e-6bfb-4534-816c-d73da977fa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:07.491057 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4f6e675e-6bfb-4534-816c-d73da977fa9b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:07.492554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4f6e675e-6bfb-4534-816c-d73da977fa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:07.522644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:07.581983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e4b40-87b5-453b-8353-ae8996875a73 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:07.582395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 438/884] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:07.599002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e675e-6bfb-4534-816c-d73da977fa9b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:07.605714 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:07.611675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb8458-67db-450f-bcba-428f3f80d4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:07.612043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb8458-67db-450f-bcba-428f3f80d4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:07.612333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb8458-67db-450f-bcba-428f3f80d4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:07.612617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb8458-67db-450f-bcba-428f3f80d4d6 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:07.613311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb8458-67db-450f-bcba-428f3f80d4d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:07.614709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb8458-67db-450f-bcba-428f3f80d4d6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:07.621673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 447/885] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 06:18:07 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:18:07.719092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e675e-6bfb-4534-816c-d73da977fa9b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:07.757134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf272a-57bc-40f2-a437-dc2772e44e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:07.758353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf272a-57bc-40f2-a437-dc2772e44e95 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:07.758846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf272a-57bc-40f2-a437-dc2772e44e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:07.759326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf272a-57bc-40f2-a437-dc2772e44e95 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:07.760199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf272a-57bc-40f2-a437-dc2772e44e95 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:07.762298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf272a-57bc-40f2-a437-dc2772e44e95 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e4cf272a-57bc-40f2-a437-dc2772e44e95 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:07.776171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:07.777250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 06:18:07.778227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:07.785303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 439/886] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:18:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:07.788193 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb8458-67db-450f-bcba-428f3f80d4d6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:07.897587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb8458-67db-450f-bcba-428f3f80d4d6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:07.938749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249065d-9773-4964-b281-0d20c8307b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:07.939445 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249065d-9773-4964-b281-0d20c8307b8f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:07.939734 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249065d-9773-4964-b281-0d20c8307b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:07.940019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249065d-9773-4964-b281-0d20c8307b8f None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:07.940746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249065d-9773-4964-b281-0d20c8307b8f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:07.941061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249065d-9773-4964-b281-0d20c8307b8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:07.942481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249065d-9773-4964-b281-0d20c8307b8f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:07.945834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3315d-9e68-4ff3-9d5c-ff4e47b9304d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:07.946317 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3315d-9e68-4ff3-9d5c-ff4e47b9304d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:07.946556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3315d-9e68-4ff3-9d5c-ff4e47b9304d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:07.948168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 440/888] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:07.948280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 448/888] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:18:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:07.953110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e77cce-2088-48ae-b3a2-84097f7e82b0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:07.953398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e77cce-2088-48ae-b3a2-84097f7e82b0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:07.953710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e77cce-2088-48ae-b3a2-84097f7e82b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:07.955088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249065d-9773-4964-b281-0d20c8307b8f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:07.978048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-90e77cce-2088-48ae-b3a2-84097f7e82b0 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:07.979347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-90e77cce-2088-48ae-b3a2-84097f7e82b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:07.997100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:08.028713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:08.075864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249065d-9773-4964-b281-0d20c8307b8f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:08.082192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 441/889] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:08.100045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e77cce-2088-48ae-b3a2-84097f7e82b0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:08.112329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be5691-c226-44f6-86e1-cef9f02b47fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:08.113008 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be5691-c226-44f6-86e1-cef9f02b47fb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:08.113294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be5691-c226-44f6-86e1-cef9f02b47fb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:08.113576 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be5691-c226-44f6-86e1-cef9f02b47fb None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:08.114342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be5691-c226-44f6-86e1-cef9f02b47fb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:08.114655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be5691-c226-44f6-86e1-cef9f02b47fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:08.116038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be5691-c226-44f6-86e1-cef9f02b47fb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:08.121423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 449/890] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:18:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:08.127469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be5691-c226-44f6-86e1-cef9f02b47fb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:08.222850 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e77cce-2088-48ae-b3a2-84097f7e82b0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:08.249742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be5691-c226-44f6-86e1-cef9f02b47fb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:08.262648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe8d3a-f5d5-4e1e-97b5-5d154b8e4489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:08.263762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe8d3a-f5d5-4e1e-97b5-5d154b8e4489 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:08.264210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe8d3a-f5d5-4e1e-97b5-5d154b8e4489 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:08.264756 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe8d3a-f5d5-4e1e-97b5-5d154b8e4489 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:08.265685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe8d3a-f5d5-4e1e-97b5-5d154b8e4489 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:08.267448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe8d3a-f5d5-4e1e-97b5-5d154b8e4489 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:08.274860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c6fe8d3a-f5d5-4e1e-97b5-5d154b8e4489 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:08.282670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:08.283659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:08.284536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:08.287487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 442/891] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 06:18:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:08.291251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2407f-5351-4e07-8ce2-4b1a4be6cf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:08.292231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2407f-5351-4e07-8ce2-4b1a4be6cf1c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:08.292744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2407f-5351-4e07-8ce2-4b1a4be6cf1c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:08.293320 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe8d3a-f5d5-4e1e-97b5-5d154b8e4489 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:08.293529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2407f-5351-4e07-8ce2-4b1a4be6cf1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:08.294619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2407f-5351-4e07-8ce2-4b1a4be6cf1c None admin] RBAC: Adding query filter params (interface=public, service_id=0c1f255fd90f43b08f0a1b862a99782e, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:08.295149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2407f-5351-4e07-8ce2-4b1a4be6cf1c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:08.298087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2407f-5351-4e07-8ce2-4b1a4be6cf1c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:08.309568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 450/892] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 06:18:08 2022] GET /identity/v3/endpoints?service_id=0c1f255fd90f43b08f0a1b862a99782e&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:08.311488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2407f-5351-4e07-8ce2-4b1a4be6cf1c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:08.315059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:08.413571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe8d3a-f5d5-4e1e-97b5-5d154b8e4489 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:08.432341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2407f-5351-4e07-8ce2-4b1a4be6cf1c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:08.451254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5adc8-1e7f-4c74-9436-e3dc7a12d3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:08.452353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5adc8-1e7f-4c74-9436-e3dc7a12d3fa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:08.453130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5adc8-1e7f-4c74-9436-e3dc7a12d3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:08.453594 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5adc8-1e7f-4c74-9436-e3dc7a12d3fa None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:08.454520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5adc8-1e7f-4c74-9436-e3dc7a12d3fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:08.455033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5adc8-1e7f-4c74-9436-e3dc7a12d3fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:08.457287 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5adc8-1e7f-4c74-9436-e3dc7a12d3fa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:08.464155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 443/893] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 06:18:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:08.471097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5adc8-1e7f-4c74-9436-e3dc7a12d3fa None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:08.475885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1bf93-2ebf-4ebf-a2f3-8fa0658d65fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:08.476560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1bf93-2ebf-4ebf-a2f3-8fa0658d65fc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:08.476894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1bf93-2ebf-4ebf-a2f3-8fa0658d65fc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:08.477176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1bf93-2ebf-4ebf-a2f3-8fa0658d65fc None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:08.477915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1bf93-2ebf-4ebf-a2f3-8fa0658d65fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:08.479327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1bf93-2ebf-4ebf-a2f3-8fa0658d65fc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:08.493232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 451/894] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 06:18:08 2022] GET /identity/v3/services => generated 1268 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:18:08.569322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:08.595388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5adc8-1e7f-4c74-9436-e3dc7a12d3fa None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:08.637529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6b283-ff08-42ea-8df0-9f1b00605832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:08.638643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6b283-ff08-42ea-8df0-9f1b00605832 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:08.639161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6b283-ff08-42ea-8df0-9f1b00605832 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:08.639660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6b283-ff08-42ea-8df0-9f1b00605832 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:08.640595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6b283-ff08-42ea-8df0-9f1b00605832 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:08.641108 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6b283-ff08-42ea-8df0-9f1b00605832 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:08.642742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6b283-ff08-42ea-8df0-9f1b00605832 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:08.649140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 444/895] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 06:18:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:08.759586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24450124-5276-4eaa-ae11-6e72629cb1d8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:08.759874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24450124-5276-4eaa-ae11-6e72629cb1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:08.760152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24450124-5276-4eaa-ae11-6e72629cb1d8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:08.762388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 452/896] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:08.767812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be311ed4-35b2-454e-931d-99b72e241891 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:08.768426 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be311ed4-35b2-454e-931d-99b72e241891 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:08.768910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be311ed4-35b2-454e-931d-99b72e241891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:08.787503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-be311ed4-35b2-454e-931d-99b72e241891 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:08.788946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-be311ed4-35b2-454e-931d-99b72e241891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:08.866089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 445/897] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:08.896878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24450124-5276-4eaa-ae11-6e72629cb1d8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:08.952519 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:09.007226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24450124-5276-4eaa-ae11-6e72629cb1d8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:09.047755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12aeec-0488-4566-aeaf-c74df8a72aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:09.048588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12aeec-0488-4566-aeaf-c74df8a72aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:09.048869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12aeec-0488-4566-aeaf-c74df8a72aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:09.049147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12aeec-0488-4566-aeaf-c74df8a72aa4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:09.064448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12aeec-0488-4566-aeaf-c74df8a72aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ea12aeec-0488-4566-aeaf-c74df8a72aa4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:09.072860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:09.073920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:09.074838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:09.077537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 453/898] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 06:18:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:09.083550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be311ed4-35b2-454e-931d-99b72e241891 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:09.156147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:09.189654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be311ed4-35b2-454e-931d-99b72e241891 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:09.225408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42771877-c1e7-4a0d-ab83-0052dedea197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:09.226529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42771877-c1e7-4a0d-ab83-0052dedea197 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:09.227028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42771877-c1e7-4a0d-ab83-0052dedea197 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:09.227549 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42771877-c1e7-4a0d-ab83-0052dedea197 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:09.228549 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42771877-c1e7-4a0d-ab83-0052dedea197 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:09.229062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42771877-c1e7-4a0d-ab83-0052dedea197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:09.231410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42771877-c1e7-4a0d-ab83-0052dedea197 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:09.244556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 446/899] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 06:18:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:09.250690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12aeec-0488-4566-aeaf-c74df8a72aa4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:09.357810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12aeec-0488-4566-aeaf-c74df8a72aa4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:09.368610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:09.402404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddcc13-786e-4518-8079-0340b93f97f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:09.403246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddcc13-786e-4518-8079-0340b93f97f1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:09.403530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddcc13-786e-4518-8079-0340b93f97f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:09.403805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddcc13-786e-4518-8079-0340b93f97f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:09.415315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddcc13-786e-4518-8079-0340b93f97f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:09.416706 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddcc13-786e-4518-8079-0340b93f97f1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e5ddcc13-786e-4518-8079-0340b93f97f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:09.435961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:09.436917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:09.437874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:09.437874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:09.437874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:09.437874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:09.437874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 454/900] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:09.444281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42771877-c1e7-4a0d-ab83-0052dedea197 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:09.553298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42771877-c1e7-4a0d-ab83-0052dedea197 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:09.588382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e2c02-a997-4e67-9895-29da77f22d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:09.589050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e2c02-a997-4e67-9895-29da77f22d7a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:09.589357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e2c02-a997-4e67-9895-29da77f22d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:09.589642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e2c02-a997-4e67-9895-29da77f22d7a None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:09.590569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e2c02-a997-4e67-9895-29da77f22d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:09.590881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e2c02-a997-4e67-9895-29da77f22d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:09.592242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e2c02-a997-4e67-9895-29da77f22d7a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:09.593693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:09.604853 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 447/901] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:09.610641 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddcc13-786e-4518-8079-0340b93f97f1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:09.719517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddcc13-786e-4518-8079-0340b93f97f1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:09.757536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cab9e4-c196-4aca-8e24-8e0666bede66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:09.758387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cab9e4-c196-4aca-8e24-8e0666bede66 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:09.758670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cab9e4-c196-4aca-8e24-8e0666bede66 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:09.758948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cab9e4-c196-4aca-8e24-8e0666bede66 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:09.767237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cab9e4-c196-4aca-8e24-8e0666bede66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:09.768698 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cab9e4-c196-4aca-8e24-8e0666bede66 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-42cab9e4-c196-4aca-8e24-8e0666bede66 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:18:09.784188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:09.785172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:09.786142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:09.786142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:09.786142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:18:09.786142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:18:09.786142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:18:09.786142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:18:09.786142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:18:09.786142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:18:09.786142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:18:09.786142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:09.786142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 455/902] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 06:18:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:09.788599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68b181-6f6b-4d12-9eb4-12c476ddca08 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:09.788892 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68b181-6f6b-4d12-9eb4-12c476ddca08 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:09.789175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68b181-6f6b-4d12-9eb4-12c476ddca08 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:09.790771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 448/903] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:09.794604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cab9e4-c196-4aca-8e24-8e0666bede66 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:09.796670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8194832-3ef8-4874-8e15-3da6d2cf79d8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:09.796959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8194832-3ef8-4874-8e15-3da6d2cf79d8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:09.797237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8194832-3ef8-4874-8e15-3da6d2cf79d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:09.815395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f8194832-3ef8-4874-8e15-3da6d2cf79d8 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:09.816607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f8194832-3ef8-4874-8e15-3da6d2cf79d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:09.850364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:09.898196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 449/904] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:09.902263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cab9e4-c196-4aca-8e24-8e0666bede66 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:09.914176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8194832-3ef8-4874-8e15-3da6d2cf79d8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:09.938517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bcd4c-f669-46c3-957c-6e2fe14c3228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:09.939257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bcd4c-f669-46c3-957c-6e2fe14c3228 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:09.939539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bcd4c-f669-46c3-957c-6e2fe14c3228 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:09.939817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bcd4c-f669-46c3-957c-6e2fe14c3228 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:09.940719 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bcd4c-f669-46c3-957c-6e2fe14c3228 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:09.941035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bcd4c-f669-46c3-957c-6e2fe14c3228 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:09.942514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bcd4c-f669-46c3-957c-6e2fe14c3228 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:09.960806 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 456/905] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 06:18:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:09.967292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bcd4c-f669-46c3-957c-6e2fe14c3228 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:10.027671 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8194832-3ef8-4874-8e15-3da6d2cf79d8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:10.057968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:10.069212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4386e5-b41c-455d-aee9-fffbfa08ef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:10.070305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4386e5-b41c-455d-aee9-fffbfa08ef48 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.070787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4386e5-b41c-455d-aee9-fffbfa08ef48 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.071247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4386e5-b41c-455d-aee9-fffbfa08ef48 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.072093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4386e5-b41c-455d-aee9-fffbfa08ef48 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:10.073626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4386e5-b41c-455d-aee9-fffbfa08ef48 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:10.086375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2d4386e5-b41c-455d-aee9-fffbfa08ef48 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:10.087301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:10.088555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 06:18:10.089918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:10.098020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 450/906] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Jul 21 06:18:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:10.098170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bcd4c-f669-46c3-957c-6e2fe14c3228 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:10.099683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4386e5-b41c-455d-aee9-fffbfa08ef48 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:10.137035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5a0c7-ef0e-40ae-8e9b-13eba6d11c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:10.137690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5a0c7-ef0e-40ae-8e9b-13eba6d11c27 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.138013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5a0c7-ef0e-40ae-8e9b-13eba6d11c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.138298 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5a0c7-ef0e-40ae-8e9b-13eba6d11c27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.216100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4386e5-b41c-455d-aee9-fffbfa08ef48 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:10.231355 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5a0c7-ef0e-40ae-8e9b-13eba6d11c27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:10.233541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5a0c7-ef0e-40ae-8e9b-13eba6d11c27 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:10.253890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690be20-7a49-444b-947a-d3a5eead81b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:10.254935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690be20-7a49-444b-947a-d3a5eead81b1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.255444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690be20-7a49-444b-947a-d3a5eead81b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.255965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690be20-7a49-444b-947a-d3a5eead81b1 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.256882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690be20-7a49-444b-947a-d3a5eead81b1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:10.257392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690be20-7a49-444b-947a-d3a5eead81b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:10.258981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690be20-7a49-444b-947a-d3a5eead81b1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:10.269480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 451/907] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 06:18:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:10.270621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690be20-7a49-444b-947a-d3a5eead81b1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:10.329243 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:10.340507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:10.371985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 457/908] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:09 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 06:18:10.377151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5a0c7-ef0e-40ae-8e9b-13eba6d11c27 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:10.389106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690be20-7a49-444b-947a-d3a5eead81b1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:10.424450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fba30f-4abf-4aaa-b2f4-4d9003f9faa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:10.425528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fba30f-4abf-4aaa-b2f4-4d9003f9faa8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.425875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fba30f-4abf-4aaa-b2f4-4d9003f9faa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.426270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fba30f-4abf-4aaa-b2f4-4d9003f9faa8 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.427055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fba30f-4abf-4aaa-b2f4-4d9003f9faa8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:10.427437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fba30f-4abf-4aaa-b2f4-4d9003f9faa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:10.428891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fba30f-4abf-4aaa-b2f4-4d9003f9faa8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:10.434750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 452/909] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 06:18:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:10.443423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fba30f-4abf-4aaa-b2f4-4d9003f9faa8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:10.496085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5a0c7-ef0e-40ae-8e9b-13eba6d11c27 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:10.534053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3be2f-12e3-4493-86e4-5af961ffa2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:10.535294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3be2f-12e3-4493-86e4-5af961ffa2bf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.535294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3be2f-12e3-4493-86e4-5af961ffa2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.535488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3be2f-12e3-4493-86e4-5af961ffa2bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.546169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3be2f-12e3-4493-86e4-5af961ffa2bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:10.547845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3be2f-12e3-4493-86e4-5af961ffa2bf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-10f3be2f-12e3-4493-86e4-5af961ffa2bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:10.566055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:10.567004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:10.567945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:10.567945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:10.567945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:10.567945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:10.567945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 458/910] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:10.574155 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3be2f-12e3-4493-86e4-5af961ffa2bf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:10.583909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fba30f-4abf-4aaa-b2f4-4d9003f9faa8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:10.602536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:10.609194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:10.624142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8485151-e2d5-4528-a9c4-401f103dbf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:10.625156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8485151-e2d5-4528-a9c4-401f103dbf45 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.625653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8485151-e2d5-4528-a9c4-401f103dbf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.626245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8485151-e2d5-4528-a9c4-401f103dbf45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.627295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8485151-e2d5-4528-a9c4-401f103dbf45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:10.628848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8485151-e2d5-4528-a9c4-401f103dbf45 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:10.653004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 453/911] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 06:18:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:18:10.697253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3be2f-12e3-4493-86e4-5af961ffa2bf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:10.732406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce03b67-f592-432e-b447-8e2b5779c4ab None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.732777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce03b67-f592-432e-b447-8e2b5779c4ab None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.732977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce03b67-f592-432e-b447-8e2b5779c4ab None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.734673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 454/912] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:18:10.740404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7345caa-5ea6-4726-8fb4-316af284e14c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.741155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7345caa-5ea6-4726-8fb4-316af284e14c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.741649 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7345caa-5ea6-4726-8fb4-316af284e14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.744089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b6160-5080-4d91-8614-f0ffc358ed9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:10.744778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b6160-5080-4d91-8614-f0ffc358ed9f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.745067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b6160-5080-4d91-8614-f0ffc358ed9f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.745347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b6160-5080-4d91-8614-f0ffc358ed9f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.746231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b6160-5080-4d91-8614-f0ffc358ed9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:10.746546 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b6160-5080-4d91-8614-f0ffc358ed9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:10.747960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b6160-5080-4d91-8614-f0ffc358ed9f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:10.761033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 459/913] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:10.767502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b6160-5080-4d91-8614-f0ffc358ed9f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:10.768076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b7345caa-5ea6-4726-8fb4-316af284e14c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:10.769563 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b7345caa-5ea6-4726-8fb4-316af284e14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:10.848699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 455/914] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:10.850352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5cf0a-29b8-4ce8-bd26-0631bbd987e3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.850804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5cf0a-29b8-4ce8-bd26-0631bbd987e3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.851234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5cf0a-29b8-4ce8-bd26-0631bbd987e3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.852642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 456/915] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:10.863265 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008cc00-f24c-46e3-a228-244a4e82137c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.863557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008cc00-f24c-46e3-a228-244a4e82137c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.863835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008cc00-f24c-46e3-a228-244a4e82137c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.876363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:10.881819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9008cc00-f24c-46e3-a228-244a4e82137c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:10.883697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9008cc00-f24c-46e3-a228-244a4e82137c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:10.891595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b6160-5080-4d91-8614-f0ffc358ed9f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:10.931553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443ba5f-fead-4459-9b1f-8372ff068155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:10.932386 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443ba5f-fead-4459-9b1f-8372ff068155 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:10.932673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443ba5f-fead-4459-9b1f-8372ff068155 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:10.932983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443ba5f-fead-4459-9b1f-8372ff068155 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:10.940235 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443ba5f-fead-4459-9b1f-8372ff068155 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:10.941687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443ba5f-fead-4459-9b1f-8372ff068155 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:10.954559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 457/916] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:10.955408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008cc00-f24c-46e3-a228-244a4e82137c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8443ba5f-fead-4459-9b1f-8372ff068155 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:18:10.956201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:10.957051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:10.957928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:18:10.957928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:18:10.957928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:18:10.957928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:18:10.957928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:18:10.957928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:18:10.957928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:10.957928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:10.957928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 460/917] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:10.963650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443ba5f-fead-4459-9b1f-8372ff068155 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:10.993697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:11.077314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443ba5f-fead-4459-9b1f-8372ff068155 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:11.077497 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008cc00-f24c-46e3-a228-244a4e82137c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:11.115778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95365a-fdea-43cd-833e-b434d734d81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:11.116398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588eced-9529-4228-aa73-9cf6a1660af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:11.116513 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95365a-fdea-43cd-833e-b434d734d81d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:11.116784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95365a-fdea-43cd-833e-b434d734d81d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:11.116885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588eced-9529-4228-aa73-9cf6a1660af1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:11.117060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95365a-fdea-43cd-833e-b434d734d81d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:11.117163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588eced-9529-4228-aa73-9cf6a1660af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:11.117381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588eced-9529-4228-aa73-9cf6a1660af1 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:11.117718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95365a-fdea-43cd-833e-b434d734d81d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:11.118197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588eced-9529-4228-aa73-9cf6a1660af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:11.118504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588eced-9529-4228-aa73-9cf6a1660af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:11.119133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95365a-fdea-43cd-833e-b434d734d81d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:11.119908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588eced-9529-4228-aa73-9cf6a1660af1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fc95365a-fdea-43cd-833e-b434d734d81d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:11.131232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:11.132217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 06:18:11.133569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:11.134652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 461/918] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:18:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:11.134652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588eced-9529-4228-aa73-9cf6a1660af1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:11.141991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 458/919] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:18:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:11.141991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95365a-fdea-43cd-833e-b434d734d81d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:11.151501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:11.244275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:11.246616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588eced-9529-4228-aa73-9cf6a1660af1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:11.252541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95365a-fdea-43cd-833e-b434d734d81d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:11.282104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186212b8-8da3-4b51-a825-eaa4c35b67fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:11.282892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186212b8-8da3-4b51-a825-eaa4c35b67fa None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:11.283186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186212b8-8da3-4b51-a825-eaa4c35b67fa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:11.283471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186212b8-8da3-4b51-a825-eaa4c35b67fa None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:11.284199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186212b8-8da3-4b51-a825-eaa4c35b67fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:11.285586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186212b8-8da3-4b51-a825-eaa4c35b67fa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:11.290460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974fb2a-cda5-4a2c-94d3-04d423726eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:11.291119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974fb2a-cda5-4a2c-94d3-04d423726eee None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:11.291460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974fb2a-cda5-4a2c-94d3-04d423726eee None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:11.291795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974fb2a-cda5-4a2c-94d3-04d423726eee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:11.292689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974fb2a-cda5-4a2c-94d3-04d423726eee None admin] RBAC: Adding query filter params (role.id=d726d771b3d34d378b43824cbfec19c9, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=e0ff633636ec403780dd4bd21e0fa65a) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:11.293044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974fb2a-cda5-4a2c-94d3-04d423726eee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:11.294599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974fb2a-cda5-4a2c-94d3-04d423726eee None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:11.306362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 462/920] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:18:11 2022] POST /identity/v3/services => generated 241 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:18:11.306362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186212b8-8da3-4b51-a825-eaa4c35b67fa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:11.306628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 459/921] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:18:11 2022] GET /identity/v3/role_assignments?user.id=e0ff633636ec403780dd4bd21e0fa65a&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=d726d771b3d34d378b43824cbfec19c9 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:11.389797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:11.454285 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186212b8-8da3-4b51-a825-eaa4c35b67fa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:11.494949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a9ab9-49ee-491a-87e7-9ce1ae08cfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:11.496902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a9ab9-49ee-491a-87e7-9ce1ae08cfda None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:11.497396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a9ab9-49ee-491a-87e7-9ce1ae08cfda None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:11.497905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a9ab9-49ee-491a-87e7-9ce1ae08cfda None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:11.498990 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a9ab9-49ee-491a-87e7-9ce1ae08cfda None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:11.499518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a9ab9-49ee-491a-87e7-9ce1ae08cfda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:11.501113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a9ab9-49ee-491a-87e7-9ce1ae08cfda None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:11.510047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 463/922] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:18:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:11.514897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974fb2a-cda5-4a2c-94d3-04d423726eee None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:11.622439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974fb2a-cda5-4a2c-94d3-04d423726eee None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:11.658861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62147677-4f90-4ba0-b07b-884c7b08f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:11.659857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62147677-4f90-4ba0-b07b-884c7b08f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:11.660346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62147677-4f90-4ba0-b07b-884c7b08f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:11.660839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62147677-4f90-4ba0-b07b-884c7b08f9d6 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:11.661750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62147677-4f90-4ba0-b07b-884c7b08f9d6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:11.662292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62147677-4f90-4ba0-b07b-884c7b08f9d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:11.663854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62147677-4f90-4ba0-b07b-884c7b08f9d6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:11.674037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 460/923] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 06:18:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:11.679045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a9ab9-49ee-491a-87e7-9ce1ae08cfda None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:11.786927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a9ab9-49ee-491a-87e7-9ce1ae08cfda None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:11.818306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:11.823533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49296c25-0398-4093-b592-71ae07045cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:11.824659 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296c25-0398-4093-b592-71ae07045cac None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:11.825127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296c25-0398-4093-b592-71ae07045cac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:11.825528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296c25-0398-4093-b592-71ae07045cac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:11.826329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49296c25-0398-4093-b592-71ae07045cac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:11.827715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49296c25-0398-4093-b592-71ae07045cac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:11.852717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 464/924] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 06:18:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:18:12.810860 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91579f46-ecbf-42ec-9809-3bc827008885 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:12.811583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91579f46-ecbf-42ec-9809-3bc827008885 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:12.811583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91579f46-ecbf-42ec-9809-3bc827008885 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:12.813056 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 461/925] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:12.818560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a52fc-4825-4daa-8d5d-7fad283dba68 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:12.818960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a52fc-4825-4daa-8d5d-7fad283dba68 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:12.819128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a52fc-4825-4daa-8d5d-7fad283dba68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:12.835893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5e7a52fc-4825-4daa-8d5d-7fad283dba68 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:12.837073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5e7a52fc-4825-4daa-8d5d-7fad283dba68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:12.910728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 465/926] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:12.932834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91579f46-ecbf-42ec-9809-3bc827008885 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:12.960409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:13.047556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91579f46-ecbf-42ec-9809-3bc827008885 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:13.084390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072f157-f3cc-4667-bc7c-f333577a6940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:13.085115 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072f157-f3cc-4667-bc7c-f333577a6940 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:13.085398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072f157-f3cc-4667-bc7c-f333577a6940 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:13.085675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072f157-f3cc-4667-bc7c-f333577a6940 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:13.086382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072f157-f3cc-4667-bc7c-f333577a6940 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:13.087749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072f157-f3cc-4667-bc7c-f333577a6940 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2072f157-f3cc-4667-bc7c-f333577a6940 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:13.100434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:13.101427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:13.102284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 462/927] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:13.110039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a52fc-4825-4daa-8d5d-7fad283dba68 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:13.212509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a52fc-4825-4daa-8d5d-7fad283dba68 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:13.234331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:13.252672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f09b9-9e47-4a33-9364-a6b2652c5e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:13.253377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f09b9-9e47-4a33-9364-a6b2652c5e29 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:13.253665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f09b9-9e47-4a33-9364-a6b2652c5e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:13.253957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f09b9-9e47-4a33-9364-a6b2652c5e29 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:13.254890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f09b9-9e47-4a33-9364-a6b2652c5e29 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:13.255249 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f09b9-9e47-4a33-9364-a6b2652c5e29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:13.256628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f09b9-9e47-4a33-9364-a6b2652c5e29 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:13.262882 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 466/928] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:18:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:13.268447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072f157-f3cc-4667-bc7c-f333577a6940 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:13.362481 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:13.377251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072f157-f3cc-4667-bc7c-f333577a6940 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:13.411541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b488-cc57-4b34-b9f6-a438ec56fe3d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:13.411849 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b488-cc57-4b34-b9f6-a438ec56fe3d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:13.412102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b488-cc57-4b34-b9f6-a438ec56fe3d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:13.413745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 467/929] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:18:13.418874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407e4e8-0db7-4951-a661-2d70390c55dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:13.419563 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e4e8-0db7-4951-a661-2d70390c55dc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:13.419849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e4e8-0db7-4951-a661-2d70390c55dc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:13.420128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e4e8-0db7-4951-a661-2d70390c55dc None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:13.420844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407e4e8-0db7-4951-a661-2d70390c55dc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:13.421152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407e4e8-0db7-4951-a661-2d70390c55dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:13.422994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407e4e8-0db7-4951-a661-2d70390c55dc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:13.425133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d413a-d431-4adb-a618-ebb350fd9790 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:13.425822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d413a-d431-4adb-a618-ebb350fd9790 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:13.426375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d413a-d431-4adb-a618-ebb350fd9790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:13.427864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 463/930] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:18:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:13.450399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-425d413a-d431-4adb-a618-ebb350fd9790 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:13.452028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-425d413a-d431-4adb-a618-ebb350fd9790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:13.491090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8bd0f-faaa-444a-b9ef-6757dcec7d6e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:13.491539 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8bd0f-faaa-444a-b9ef-6757dcec7d6e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:13.491920 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8bd0f-faaa-444a-b9ef-6757dcec7d6e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:13.493807 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 464/931] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:18:13.499891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d5455-bcee-451e-a99c-e3a6c6e3fb1f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:13.500179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d5455-bcee-451e-a99c-e3a6c6e3fb1f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:13.500468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d5455-bcee-451e-a99c-e3a6c6e3fb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:13.519391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-792d5455-bcee-451e-a99c-e3a6c6e3fb1f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:13.520605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-792d5455-bcee-451e-a99c-e3a6c6e3fb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:13.546003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 468/932] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:13.565246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d413a-d431-4adb-a618-ebb350fd9790 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:13.611792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 465/933] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:13.636658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d5455-bcee-451e-a99c-e3a6c6e3fb1f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:13.701542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d413a-d431-4adb-a618-ebb350fd9790 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:13.705335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:13.729687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:13.750252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afd90b-91c1-46e9-895c-7d63ad4fcee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:13.751071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afd90b-91c1-46e9-895c-7d63ad4fcee2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:13.751368 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afd90b-91c1-46e9-895c-7d63ad4fcee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:13.751809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afd90b-91c1-46e9-895c-7d63ad4fcee2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:13.759381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d5455-bcee-451e-a99c-e3a6c6e3fb1f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:13.764031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afd90b-91c1-46e9-895c-7d63ad4fcee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a5afd90b-91c1-46e9-895c-7d63ad4fcee2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:13.771535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:13.772745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:13.773921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:13.778821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 469/934] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 06:18:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:13.779834 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afd90b-91c1-46e9-895c-7d63ad4fcee2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:13.810913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb7dc0-adfc-43c0-ab52-12b5a6b91c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:13.812947 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb7dc0-adfc-43c0-ab52-12b5a6b91c18 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:13.813530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb7dc0-adfc-43c0-ab52-12b5a6b91c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:13.814168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb7dc0-adfc-43c0-ab52-12b5a6b91c18 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:13.815235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb7dc0-adfc-43c0-ab52-12b5a6b91c18 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:13.817463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb7dc0-adfc-43c0-ab52-12b5a6b91c18 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f5fb7dc0-adfc-43c0-ab52-12b5a6b91c18 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:13.834651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:13.835867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:13.836896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:13.840174 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 466/935] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 06:18:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:13.846985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb7dc0-adfc-43c0-ab52-12b5a6b91c18 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:13.910250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afd90b-91c1-46e9-895c-7d63ad4fcee2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:13.956668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15aa88-3fc1-47cd-abbd-a5e8e68b32e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:13.957593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15aa88-3fc1-47cd-abbd-a5e8e68b32e1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:13.958057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15aa88-3fc1-47cd-abbd-a5e8e68b32e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:13.959095 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15aa88-3fc1-47cd-abbd-a5e8e68b32e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:13.960297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15aa88-3fc1-47cd-abbd-a5e8e68b32e1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:13.960757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15aa88-3fc1-47cd-abbd-a5e8e68b32e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:13.963199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15aa88-3fc1-47cd-abbd-a5e8e68b32e1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:13.971682 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb7dc0-adfc-43c0-ab52-12b5a6b91c18 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:13.975719 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 470/936] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 06:18:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:13.980933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15aa88-3fc1-47cd-abbd-a5e8e68b32e1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:14.023003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:14.025759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938a607-f658-441f-8cfa-ded48da60b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:14.026471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938a607-f658-441f-8cfa-ded48da60b3a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.027192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938a607-f658-441f-8cfa-ded48da60b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.027614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938a607-f658-441f-8cfa-ded48da60b3a None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.028338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938a607-f658-441f-8cfa-ded48da60b3a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:14.028650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938a607-f658-441f-8cfa-ded48da60b3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:14.030509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938a607-f658-441f-8cfa-ded48da60b3a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:14.042148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 467/937] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 06:18:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:14.043644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ece696-5260-47c1-8119-0a3ef0021da4 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.043935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ece696-5260-47c1-8119-0a3ef0021da4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.044223 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ece696-5260-47c1-8119-0a3ef0021da4 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.049423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 468/938] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:14.049423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ece696-5260-47c1-8119-0a3ef0021da4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:14.065431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:14.105978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15aa88-3fc1-47cd-abbd-a5e8e68b32e1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:14.141616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41814d89-47f6-4c8f-b1bf-0c3fc141053a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:14.142451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41814d89-47f6-4c8f-b1bf-0c3fc141053a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.142732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41814d89-47f6-4c8f-b1bf-0c3fc141053a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.143008 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41814d89-47f6-4c8f-b1bf-0c3fc141053a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.155525 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41814d89-47f6-4c8f-b1bf-0c3fc141053a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:14.157281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41814d89-47f6-4c8f-b1bf-0c3fc141053a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:14.174648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ece696-5260-47c1-8119-0a3ef0021da4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-41814d89-47f6-4c8f-b1bf-0c3fc141053a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:14.175556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:14.176401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:14.177328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:14.177328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:14.177328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:14.177328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:14.177328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 471/939] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:14.178166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d687c6-2f6c-4a63-873c-e870efdd1da7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.178267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d687c6-2f6c-4a63-873c-e870efdd1da7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.178520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d687c6-2f6c-4a63-873c-e870efdd1da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.195459 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-76d687c6-2f6c-4a63-873c-e870efdd1da7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:14.196657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-76d687c6-2f6c-4a63-873c-e870efdd1da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:14.219587 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c9af8-4b14-4714-9318-e45a9b22254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:14.220514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9af8-4b14-4714-9318-e45a9b22254d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.220857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9af8-4b14-4714-9318-e45a9b22254d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.221069 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9af8-4b14-4714-9318-e45a9b22254d None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.221787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c9af8-4b14-4714-9318-e45a9b22254d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:14.222113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c9af8-4b14-4714-9318-e45a9b22254d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:14.223461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c9af8-4b14-4714-9318-e45a9b22254d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:14.228462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 469/940] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 06:18:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:14.229251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c9af8-4b14-4714-9318-e45a9b22254d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:14.272590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 472/941] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:14.273431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d687c6-2f6c-4a63-873c-e870efdd1da7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:14.296811 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:14.316866 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:14.345922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c9af8-4b14-4714-9318-e45a9b22254d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:14.384404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb7b32-c6d8-440d-bb65-c94e2ed9b29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:14.385071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb7b32-c6d8-440d-bb65-c94e2ed9b29f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.385358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb7b32-c6d8-440d-bb65-c94e2ed9b29f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.385731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb7b32-c6d8-440d-bb65-c94e2ed9b29f None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.386444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb7b32-c6d8-440d-bb65-c94e2ed9b29f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:14.386759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb7b32-c6d8-440d-bb65-c94e2ed9b29f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:14.388127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb7b32-c6d8-440d-bb65-c94e2ed9b29f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:14.392538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d687c6-2f6c-4a63-873c-e870efdd1da7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:14.406505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 470/942] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:14.406505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb7b32-c6d8-440d-bb65-c94e2ed9b29f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:14.429375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1fbb4-1b04-4bf8-b64f-84c844fded48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:14.430397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1fbb4-1b04-4bf8-b64f-84c844fded48 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.430876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1fbb4-1b04-4bf8-b64f-84c844fded48 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.431278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1fbb4-1b04-4bf8-b64f-84c844fded48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.432151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1fbb4-1b04-4bf8-b64f-84c844fded48 None admin] RBAC: Adding query filter params (interface=public, service_id=d7bf3c276c5a45da9886b4c0f3ea49b7, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:14.432514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1fbb4-1b04-4bf8-b64f-84c844fded48 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:14.434620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1fbb4-1b04-4bf8-b64f-84c844fded48 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:14.444437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 473/943] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 06:18:14 2022] GET /identity/v3/endpoints?service_id=d7bf3c276c5a45da9886b4c0f3ea49b7&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:14.444437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1fbb4-1b04-4bf8-b64f-84c844fded48 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:14.516846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb7b32-c6d8-440d-bb65-c94e2ed9b29f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:14.518832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:14.551587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1fbb4-1b04-4bf8-b64f-84c844fded48 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:14.554081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010be3c2-1046-4f81-af99-699410a155d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:14.554840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010be3c2-1046-4f81-af99-699410a155d0 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.555129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010be3c2-1046-4f81-af99-699410a155d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.555407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010be3c2-1046-4f81-af99-699410a155d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.556122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010be3c2-1046-4f81-af99-699410a155d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:14.557560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010be3c2-1046-4f81-af99-699410a155d0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:14.559650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: WARNING keystone.server.flask.common [None req-010be3c2-1046-4f81-af99-699410a155d0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 06:18:14.563196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:14.580819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.sql.core [None req-010be3c2-1046-4f81-af99-699410a155d0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 06:18:14.580819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 06:18:14.580819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [parameters: {'id': 'd8b140414d4f456a9708c2c75c374ed1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 06:18:14.580819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80141) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 06:18:14.590752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab92ae1-8cce-46ad-8067-00f1ecaa3333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:14.591573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab92ae1-8cce-46ad-8067-00f1ecaa3333 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.591859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab92ae1-8cce-46ad-8067-00f1ecaa3333 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.592135 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab92ae1-8cce-46ad-8067-00f1ecaa3333 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.599565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab92ae1-8cce-46ad-8067-00f1ecaa3333 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-010be3c2-1046-4f81-af99-699410a155d0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d8b140414d4f456a9708c2c75c374ed1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d8b140414d4f456a9708c2c75c374ed1. Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 06:18:14.600923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 06:18:14.601679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application yield resource Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise self.value Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application session.commit() Jul 21 06:18:14.602717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.flush() Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application compat.raise_( Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 06:18:14.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 06:18:14.604527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 06:18:14.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [parameters: {'id': 'd8b140414d4f456a9708c2c75c374ed1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:14.609058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:14.611085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:14.613104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:14.613104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 06:18:14.613104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 06:18:14.613104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:14.613104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:14.613104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 06:18:14.613104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 06:18:14.613104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d8b140414d4f456a9708c2c75c374ed1. Jul 21 06:18:14.613104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:14.615321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab92ae1-8cce-46ad-8067-00f1ecaa3333 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:14.615499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 471/944] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Jul 21 06:18:14 2022] POST /identity/v3/projects => generated 262 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:14.615499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010be3c2-1046-4f81-af99-699410a155d0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bab92ae1-8cce-46ad-8067-00f1ecaa3333 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:18:14.624221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:14.631436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 474/945] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 06:18:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:14.632764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab92ae1-8cce-46ad-8067-00f1ecaa3333 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:14.729333 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010be3c2-1046-4f81-af99-699410a155d0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:14.732228 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab92ae1-8cce-46ad-8067-00f1ecaa3333 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:14.764572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:14.766899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5c16b-c21d-4967-890a-5d05398f67cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:14.767677 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5c16b-c21d-4967-890a-5d05398f67cb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.768001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5c16b-c21d-4967-890a-5d05398f67cb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.768294 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5c16b-c21d-4967-890a-5d05398f67cb None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.768991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5c16b-c21d-4967-890a-5d05398f67cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:14.770542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456f963-8758-4f0c-abec-c225a70efa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:14.770663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5c16b-c21d-4967-890a-5d05398f67cb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:14.771339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456f963-8758-4f0c-abec-c225a70efa94 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.771631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456f963-8758-4f0c-abec-c225a70efa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.772173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456f963-8758-4f0c-abec-c225a70efa94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.779799 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456f963-8758-4f0c-abec-c225a70efa94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:14.790061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 472/946] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 06:18:14 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:18:14.790276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456f963-8758-4f0c-abec-c225a70efa94 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:14.790371 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5c16b-c21d-4967-890a-5d05398f67cb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4456f963-8758-4f0c-abec-c225a70efa94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:18:14.806777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:14.807980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:14.809075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:18:14.809075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:18:14.809075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:18:14.809075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:18:14.809075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:18:14.809075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:18:14.809075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:14.809075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:14.810352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 475/947] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:14.816156 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456f963-8758-4f0c-abec-c225a70efa94 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:14.859267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:14.917992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5c16b-c21d-4967-890a-5d05398f67cb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:14.958796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2782-cd9f-464b-887e-128897af34ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:14.959531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2782-cd9f-464b-887e-128897af34ad None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:14.959821 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2782-cd9f-464b-887e-128897af34ad None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:14.960156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2782-cd9f-464b-887e-128897af34ad None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:14.961028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f2782-cd9f-464b-887e-128897af34ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:14.961404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f2782-cd9f-464b-887e-128897af34ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:14.962661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456f963-8758-4f0c-abec-c225a70efa94 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:14.963096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f2782-cd9f-464b-887e-128897af34ad None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:14.982720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 473/948] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 06:18:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:14.988300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2782-cd9f-464b-887e-128897af34ad None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:15.002609 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65eb65-f816-4986-84bc-69bf72c944ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:15.003289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65eb65-f816-4986-84bc-69bf72c944ef None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:15.003577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65eb65-f816-4986-84bc-69bf72c944ef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:15.003856 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65eb65-f816-4986-84bc-69bf72c944ef None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:15.004625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65eb65-f816-4986-84bc-69bf72c944ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:15.004938 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65eb65-f816-4986-84bc-69bf72c944ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:15.006424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65eb65-f816-4986-84bc-69bf72c944ef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:15.026127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 476/949] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:18:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:15.110791 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2782-cd9f-464b-887e-128897af34ad None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:15.126873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:15.152608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c44c9f-ac6d-4999-932f-d5561f7333d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:15.153537 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c44c9f-ac6d-4999-932f-d5561f7333d6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:15.153895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c44c9f-ac6d-4999-932f-d5561f7333d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:15.154280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c44c9f-ac6d-4999-932f-d5561f7333d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:15.166650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c44c9f-ac6d-4999-932f-d5561f7333d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:15.168361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c44c9f-ac6d-4999-932f-d5561f7333d6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-48c44c9f-ac6d-4999-932f-d5561f7333d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:15.188172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:15.189575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:15.190757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:15.190757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:15.190757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:15.190757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:15.193968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 474/950] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:15.197843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65eb65-f816-4986-84bc-69bf72c944ef None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:15.257756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:15.312640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65eb65-f816-4986-84bc-69bf72c944ef None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:15.349345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b30ddc-709c-488f-9fe4-cd95e0379ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:15.350407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b30ddc-709c-488f-9fe4-cd95e0379ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:15.350879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b30ddc-709c-488f-9fe4-cd95e0379ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:15.351338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b30ddc-709c-488f-9fe4-cd95e0379ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:15.352275 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b30ddc-709c-488f-9fe4-cd95e0379ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:15.352770 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b30ddc-709c-488f-9fe4-cd95e0379ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:15.354352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b30ddc-709c-488f-9fe4-cd95e0379ac7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:15.367587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 477/951] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:15.373595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c44c9f-ac6d-4999-932f-d5561f7333d6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:15.480302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c44c9f-ac6d-4999-932f-d5561f7333d6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:15.516885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df44075-ad20-4c1d-8bdd-76d9eb702ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:15.518014 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df44075-ad20-4c1d-8bdd-76d9eb702ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:15.518510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df44075-ad20-4c1d-8bdd-76d9eb702ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:15.518970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df44075-ad20-4c1d-8bdd-76d9eb702ce0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:15.526667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df44075-ad20-4c1d-8bdd-76d9eb702ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:15.529922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c959619-1911-4683-89c6-a690967a3d17 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:15.530310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c959619-1911-4683-89c6-a690967a3d17 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:15.530665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c959619-1911-4683-89c6-a690967a3d17 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:15.532229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 478/952] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:18:15.532357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df44075-ad20-4c1d-8bdd-76d9eb702ce0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:15.537439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aa67a-066d-47f1-baa1-594a14041bb0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:15.537904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aa67a-066d-47f1-baa1-594a14041bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:15.538357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aa67a-066d-47f1-baa1-594a14041bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1df44075-ad20-4c1d-8bdd-76d9eb702ce0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:18:15.548700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:15.549921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:15.551047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:18:15.551047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:18:15.551047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:18:15.551047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:18:15.551047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:18:15.551047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:18:15.551047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:15.551047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:15.552319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 475/953] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:15.557503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df44075-ad20-4c1d-8bdd-76d9eb702ce0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:15.559475 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-283aa67a-066d-47f1-baa1-594a14041bb0 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:15.560833 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-283aa67a-066d-47f1-baa1-594a14041bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:15.586549 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:15.652715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 479/954] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:15.673572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283aa67a-066d-47f1-baa1-594a14041bb0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:15.674426 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df44075-ad20-4c1d-8bdd-76d9eb702ce0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:15.717215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab73950-c1be-4e4e-bfee-56fa28644df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:15.718527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab73950-c1be-4e4e-bfee-56fa28644df4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:15.718966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab73950-c1be-4e4e-bfee-56fa28644df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:15.719418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab73950-c1be-4e4e-bfee-56fa28644df4 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:15.720728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab73950-c1be-4e4e-bfee-56fa28644df4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:15.721194 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab73950-c1be-4e4e-bfee-56fa28644df4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:15.721526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:15.723134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab73950-c1be-4e4e-bfee-56fa28644df4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:15.739253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 476/955] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:15.744667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab73950-c1be-4e4e-bfee-56fa28644df4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:15.790020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283aa67a-066d-47f1-baa1-594a14041bb0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:15.829745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c9fe1-0835-4c3d-bb44-72c4a2551685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:15.830850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c9fe1-0835-4c3d-bb44-72c4a2551685 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:15.831342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c9fe1-0835-4c3d-bb44-72c4a2551685 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:15.831979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c9fe1-0835-4c3d-bb44-72c4a2551685 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:15.832998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c9fe1-0835-4c3d-bb44-72c4a2551685 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:15.834811 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c9fe1-0835-4c3d-bb44-72c4a2551685 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:15.852221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 480/956] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:18:15 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:18:15.853773 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:15.890241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab73950-c1be-4e4e-bfee-56fa28644df4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:15.943635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fbbea-7891-4cce-9c7e-8f3983fd3206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:15.945050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fbbea-7891-4cce-9c7e-8f3983fd3206 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:15.945584 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fbbea-7891-4cce-9c7e-8f3983fd3206 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:15.946118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fbbea-7891-4cce-9c7e-8f3983fd3206 None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/e0ff633636ec403780dd4bd21e0fa65a/roles/d726d771b3d34d378b43824cbfec19c9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:15.949882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fbbea-7891-4cce-9c7e-8f3983fd3206 None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=e0ff633636ec403780dd4bd21e0fa65a, role_id=d726d771b3d34d378b43824cbfec19c9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:15.951819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fbbea-7891-4cce-9c7e-8f3983fd3206 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:15.972043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 477/957] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:18:15 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/e0ff633636ec403780dd4bd21e0fa65a/roles/d726d771b3d34d378b43824cbfec19c9 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:18:16.972310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43253b4d-b7e4-42a8-a4ff-6b74ec5c74c9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:16.972860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43253b4d-b7e4-42a8-a4ff-6b74ec5c74c9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:16.972860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43253b4d-b7e4-42a8-a4ff-6b74ec5c74c9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:16.974765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 481/958] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:16.980283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0d806-4c39-46eb-b5df-8b30e0ae541c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:16.980697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0d806-4c39-46eb-b5df-8b30e0ae541c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:16.980929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0d806-4c39-46eb-b5df-8b30e0ae541c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:16.999996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-90f0d806-4c39-46eb-b5df-8b30e0ae541c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:17.001579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-90f0d806-4c39-46eb-b5df-8b30e0ae541c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:17.082065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 478/959] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:17.098731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43253b4d-b7e4-42a8-a4ff-6b74ec5c74c9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:17.209899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43253b4d-b7e4-42a8-a4ff-6b74ec5c74c9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:17.211491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:17.246123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0fd59-afb7-482d-8a1d-341154686bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:17.247025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0fd59-afb7-482d-8a1d-341154686bfd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:17.247375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0fd59-afb7-482d-8a1d-341154686bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:17.248001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0fd59-afb7-482d-8a1d-341154686bfd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:17.248672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0fd59-afb7-482d-8a1d-341154686bfd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:17.250075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0fd59-afb7-482d-8a1d-341154686bfd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b4f0fd59-afb7-482d-8a1d-341154686bfd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:17.262738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:17.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:17.264695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 482/960] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 06:18:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:17.265524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a5a45-d4dd-4075-a48d-22123ab0855f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:17.265524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a5a45-d4dd-4075-a48d-22123ab0855f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:17.265701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a5a45-d4dd-4075-a48d-22123ab0855f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:17.269997 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 479/961] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:17.272159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0fd59-afb7-482d-8a1d-341154686bfd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:17.272776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1001cf0-fda7-413a-b479-c45e49c44de5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:17.273076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1001cf0-fda7-413a-b479-c45e49c44de5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:17.273373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1001cf0-fda7-413a-b479-c45e49c44de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:17.291355 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c1001cf0-fda7-413a-b479-c45e49c44de5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:17.292731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c1001cf0-fda7-413a-b479-c45e49c44de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:17.360519 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:17.365799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 480/962] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:17.382838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0fd59-afb7-482d-8a1d-341154686bfd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:17.394204 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1001cf0-fda7-413a-b479-c45e49c44de5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:17.422602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017517d-6076-4e09-8906-4226e1c57478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:17.423609 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017517d-6076-4e09-8906-4226e1c57478 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:17.424139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017517d-6076-4e09-8906-4226e1c57478 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:17.424571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017517d-6076-4e09-8906-4226e1c57478 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:17.425458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017517d-6076-4e09-8906-4226e1c57478 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:17.425919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017517d-6076-4e09-8906-4226e1c57478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:17.427702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017517d-6076-4e09-8906-4226e1c57478 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:17.434129 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 483/963] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 06:18:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:17.440465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017517d-6076-4e09-8906-4226e1c57478 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:17.473558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:17.506896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1001cf0-fda7-413a-b479-c45e49c44de5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:17.544743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c55d36-c53a-4102-99dd-e3c6c2bc54b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:17.545533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55d36-c53a-4102-99dd-e3c6c2bc54b8 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:17.545824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55d36-c53a-4102-99dd-e3c6c2bc54b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:17.546209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55d36-c53a-4102-99dd-e3c6c2bc54b8 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:17.547024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c55d36-c53a-4102-99dd-e3c6c2bc54b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:17.548461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c55d36-c53a-4102-99dd-e3c6c2bc54b8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:17.553027 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017517d-6076-4e09-8906-4226e1c57478 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:17.564706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.sql.core [None req-96c55d36-c53a-4102-99dd-e3c6c2bc54b8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 06:18:17.564706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 06:18:17.564706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [parameters: {'user_id': '551be659e87942358f1d1261e5df258f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 06:18:17.564706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80141) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-96c55d36-c53a-4102-99dd-e3c6c2bc54b8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Jul 21 06:18:17.573033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 06:18:17.574222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application yield resource Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise self.value Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application session.commit() Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 06:18:17.575327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.flush() Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application compat.raise_( Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 06:18:17.576227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 06:18:17.577120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 06:18:17.577978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [parameters: {'user_id': '551be659e87942358f1d1261e5df258f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:17.578883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 06:18:17.579813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 06:18:17.580928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 06:18:17.580928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 06:18:17.580928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 06:18:17.580928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 06:18:17.580928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 06:18:17.580928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:17.580928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 481/964] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jul 21 06:18:17 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:17.581324 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c55d36-c53a-4102-99dd-e3c6c2bc54b8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:17.600738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2d797-386f-4aea-8f57-61c1f4629393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:17.601662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d797-386f-4aea-8f57-61c1f4629393 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:17.602211 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d797-386f-4aea-8f57-61c1f4629393 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:17.602766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d797-386f-4aea-8f57-61c1f4629393 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:17.603740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2d797-386f-4aea-8f57-61c1f4629393 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:17.604202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2d797-386f-4aea-8f57-61c1f4629393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:17.606566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2d797-386f-4aea-8f57-61c1f4629393 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:17.617116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 484/965] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 06:18:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:17.618212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2d797-386f-4aea-8f57-61c1f4629393 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:17.650602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:17.698085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c55d36-c53a-4102-99dd-e3c6c2bc54b8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:17.730602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:17.738389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2d797-386f-4aea-8f57-61c1f4629393 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:17.742690 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94345638-de1c-41f7-b92b-4c77bc491d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:17.743554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345638-de1c-41f7-b92b-4c77bc491d0b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:17.743845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345638-de1c-41f7-b92b-4c77bc491d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:17.744125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345638-de1c-41f7-b92b-4c77bc491d0b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:17.752295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94345638-de1c-41f7-b92b-4c77bc491d0b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:17.754113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94345638-de1c-41f7-b92b-4c77bc491d0b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-94345638-de1c-41f7-b92b-4c77bc491d0b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:18:17.770363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:17.771286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:17.772507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:17.772507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:17.772507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:18:17.772507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:18:17.772507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:18:17.772507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:18:17.772507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:18:17.772507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:18:17.772507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 06:18:17.772507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:17.777155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 482/966] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:18:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:17.778672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94345638-de1c-41f7-b92b-4c77bc491d0b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:17.781105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da516369-4d57-44fb-9349-cb8b7baf5ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:17.782186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da516369-4d57-44fb-9349-cb8b7baf5ef5 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:17.782739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da516369-4d57-44fb-9349-cb8b7baf5ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:17.783244 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da516369-4d57-44fb-9349-cb8b7baf5ef5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:17.784206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da516369-4d57-44fb-9349-cb8b7baf5ef5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:17.785794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da516369-4d57-44fb-9349-cb8b7baf5ef5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:17.818213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 485/967] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 06:18:17 2022] POST /identity/v3/endpoints => generated 349 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:18:17.898385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94345638-de1c-41f7-b92b-4c77bc491d0b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:17.925726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:17.949319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9156915-8d22-4bfe-a986-c6b4fed46704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:17.950049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9156915-8d22-4bfe-a986-c6b4fed46704 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:17.950337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9156915-8d22-4bfe-a986-c6b4fed46704 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:17.950617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9156915-8d22-4bfe-a986-c6b4fed46704 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:17.951446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9156915-8d22-4bfe-a986-c6b4fed46704 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:17.951756 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9156915-8d22-4bfe-a986-c6b4fed46704 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:17.953209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9156915-8d22-4bfe-a986-c6b4fed46704 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:17.976508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354a29-e840-4df1-8290-266e5ce43a15 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:17.977007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354a29-e840-4df1-8290-266e5ce43a15 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:17.977085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354a29-e840-4df1-8290-266e5ce43a15 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:17.978678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 486/968] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:18:17.983957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede8b44-a075-4f90-a781-c702a320270b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:17.984251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede8b44-a075-4f90-a781-c702a320270b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:17.984537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede8b44-a075-4f90-a781-c702a320270b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:17.999247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 483/969] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 06:18:17 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:18.004959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5ede8b44-a075-4f90-a781-c702a320270b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:18.006299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5ede8b44-a075-4f90-a781-c702a320270b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:18.052528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30e10-a4a0-42ce-8c17-ecc4a6273d2a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:18.052827 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30e10-a4a0-42ce-8c17-ecc4a6273d2a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:18.053021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30e10-a4a0-42ce-8c17-ecc4a6273d2a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:18.054922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 484/970] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:18.064129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7c55e-ed15-4871-999e-808d26d22ba5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:18.064564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7c55e-ed15-4871-999e-808d26d22ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:18.064951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7c55e-ed15-4871-999e-808d26d22ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:18.084747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6cd7c55e-ed15-4871-999e-808d26d22ba5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:18.086757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6cd7c55e-ed15-4871-999e-808d26d22ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:18.096144 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 487/971] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:18.119837 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede8b44-a075-4f90-a781-c702a320270b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:18.168393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 485/972] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:18.193411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7c55e-ed15-4871-999e-808d26d22ba5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:18.216677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:18.246430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede8b44-a075-4f90-a781-c702a320270b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:18.289037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26ce99-89d2-4457-86a2-5889a78f5866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:18.290046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26ce99-89d2-4457-86a2-5889a78f5866 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:18.290436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26ce99-89d2-4457-86a2-5889a78f5866 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:18.290720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26ce99-89d2-4457-86a2-5889a78f5866 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:18.291386 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26ce99-89d2-4457-86a2-5889a78f5866 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:18.293302 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26ce99-89d2-4457-86a2-5889a78f5866 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-da26ce99-89d2-4457-86a2-5889a78f5866 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:18.307418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:18.308540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:18.309382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 488/973] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:18.315185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26ce99-89d2-4457-86a2-5889a78f5866 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:18.323355 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7c55e-ed15-4871-999e-808d26d22ba5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:18.331250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:18.363898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2fd53-4eed-48f0-96c2-1424f3de7b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:18.364718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2fd53-4eed-48f0-96c2-1424f3de7b89 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:18.365003 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2fd53-4eed-48f0-96c2-1424f3de7b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:18.365280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2fd53-4eed-48f0-96c2-1424f3de7b89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:18.375919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2fd53-4eed-48f0-96c2-1424f3de7b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-51c2fd53-4eed-48f0-96c2-1424f3de7b89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:18.384096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:18.385041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:18.385975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 486/974] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 06:18:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:18.391392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2fd53-4eed-48f0-96c2-1424f3de7b89 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:18.438601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26ce99-89d2-4457-86a2-5889a78f5866 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:18.476418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1e758-454f-414e-89ac-744856818d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:18.477087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1e758-454f-414e-89ac-744856818d79 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:18.477378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1e758-454f-414e-89ac-744856818d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:18.477659 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1e758-454f-414e-89ac-744856818d79 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:18.478399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1e758-454f-414e-89ac-744856818d79 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:18.478710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1e758-454f-414e-89ac-744856818d79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:18.480810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1e758-454f-414e-89ac-744856818d79 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:18.490414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 489/975] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:18:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:18.491705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1e758-454f-414e-89ac-744856818d79 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:18.499702 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2fd53-4eed-48f0-96c2-1424f3de7b89 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:18.504318 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:18.540460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef338-be5e-494c-a68e-f9627cb64636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:18.541535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef338-be5e-494c-a68e-f9627cb64636 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:18.542104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef338-be5e-494c-a68e-f9627cb64636 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:18.543019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef338-be5e-494c-a68e-f9627cb64636 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:18.543739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ef338-be5e-494c-a68e-f9627cb64636 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:18.544291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ef338-be5e-494c-a68e-f9627cb64636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:18.545460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ef338-be5e-494c-a68e-f9627cb64636 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:18.554448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 487/976] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 06:18:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:18.560798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef338-be5e-494c-a68e-f9627cb64636 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:18.591489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:18.605478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1e758-454f-414e-89ac-744856818d79 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:18.642192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97943f-9a7d-4470-baa8-a081ca5a4ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:18.642870 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97943f-9a7d-4470-baa8-a081ca5a4ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:18.643208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97943f-9a7d-4470-baa8-a081ca5a4ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:18.643491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97943f-9a7d-4470-baa8-a081ca5a4ff1 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:18.644244 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97943f-9a7d-4470-baa8-a081ca5a4ff1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:18.644548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97943f-9a7d-4470-baa8-a081ca5a4ff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:18.645934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97943f-9a7d-4470-baa8-a081ca5a4ff1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:18.656161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 490/977] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:18:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:18.657163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97943f-9a7d-4470-baa8-a081ca5a4ff1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:18.677048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef338-be5e-494c-a68e-f9627cb64636 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:18.715475 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c7a5d-9fec-4e60-9465-7e2b003a37e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:18.716604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7a5d-9fec-4e60-9465-7e2b003a37e2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:18.717103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7a5d-9fec-4e60-9465-7e2b003a37e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:18.717573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7a5d-9fec-4e60-9465-7e2b003a37e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:18.729252 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c7a5d-9fec-4e60-9465-7e2b003a37e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:18.731760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c7a5d-9fec-4e60-9465-7e2b003a37e2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:18.733811 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6f7c7a5d-9fec-4e60-9465-7e2b003a37e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:18.751439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:18.752390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:18.753316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:18.753316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:18.753316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:18.753316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:18.757661 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 488/978] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:18.758369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c7a5d-9fec-4e60-9465-7e2b003a37e2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:18.767015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97943f-9a7d-4470-baa8-a081ca5a4ff1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:18.803923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822d1d7-f37a-4c18-8260-54d7acf60c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:18.804965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822d1d7-f37a-4c18-8260-54d7acf60c3c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:18.805310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822d1d7-f37a-4c18-8260-54d7acf60c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:18.805668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822d1d7-f37a-4c18-8260-54d7acf60c3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:18.806490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7822d1d7-f37a-4c18-8260-54d7acf60c3c None admin] RBAC: Adding query filter params (interface=public, service_id=b057900fa79c4734acc54a28432ca7b0, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:18.806810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7822d1d7-f37a-4c18-8260-54d7acf60c3c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:18.809503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7822d1d7-f37a-4c18-8260-54d7acf60c3c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:18.815401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 491/979] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 06:18:18 2022] GET /identity/v3/endpoints?service_id=b057900fa79c4734acc54a28432ca7b0&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:18.820657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822d1d7-f37a-4c18-8260-54d7acf60c3c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:18.849349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:18.871425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c7a5d-9fec-4e60-9465-7e2b003a37e2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:18.907508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c76c7-efeb-4661-8cc0-5c8b192a33b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:18.908182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c76c7-efeb-4661-8cc0-5c8b192a33b3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:18.908460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c76c7-efeb-4661-8cc0-5c8b192a33b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:18.908747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c76c7-efeb-4661-8cc0-5c8b192a33b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:18.909484 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c76c7-efeb-4661-8cc0-5c8b192a33b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:18.909793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c76c7-efeb-4661-8cc0-5c8b192a33b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:18.911210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c76c7-efeb-4661-8cc0-5c8b192a33b3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:18.928178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 489/980] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:18.929197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c76c7-efeb-4661-8cc0-5c8b192a33b3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:18.932282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822d1d7-f37a-4c18-8260-54d7acf60c3c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:18.970355 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bbdf7-1fbb-4f18-b404-7b3324a57caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:18.971089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bbdf7-1fbb-4f18-b404-7b3324a57caa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:18.971379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bbdf7-1fbb-4f18-b404-7b3324a57caa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:18.971712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bbdf7-1fbb-4f18-b404-7b3324a57caa None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:18.972407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bbdf7-1fbb-4f18-b404-7b3324a57caa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:18.973815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bbdf7-1fbb-4f18-b404-7b3324a57caa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:18.975639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:18.993978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 492/981] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 06:18:18 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:18:19.059919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c76c7-efeb-4661-8cc0-5c8b192a33b3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:19.097455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc5eb7-2be7-468a-a990-3f0e88465608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:19.098233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc5eb7-2be7-468a-a990-3f0e88465608 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:19.098523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc5eb7-2be7-468a-a990-3f0e88465608 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:19.098806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc5eb7-2be7-468a-a990-3f0e88465608 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:19.108270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc5eb7-2be7-468a-a990-3f0e88465608 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:19.109763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc5eb7-2be7-468a-a990-3f0e88465608 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:19.112760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-93cc5eb7-2be7-468a-a990-3f0e88465608 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:18:19.127542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:19.128580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:19.134556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:19.134556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:19.134556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:18:19.134556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:18:19.134556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:18:19.134556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:18:19.134556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:18:19.134556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:18:19.134556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 06:18:19.134556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:19.134556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 490/982] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 06:18:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:19.136753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bbdf7-1fbb-4f18-b404-7b3324a57caa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:19.244498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bbdf7-1fbb-4f18-b404-7b3324a57caa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:19.279081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cb30e-375b-443d-8f69-ca5cac5f5fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:19.280060 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cb30e-375b-443d-8f69-ca5cac5f5fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:19.280497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cb30e-375b-443d-8f69-ca5cac5f5fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:19.280925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cb30e-375b-443d-8f69-ca5cac5f5fd7 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:19.281973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cb30e-375b-443d-8f69-ca5cac5f5fd7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:19.282434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cb30e-375b-443d-8f69-ca5cac5f5fd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:19.284033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cb30e-375b-443d-8f69-ca5cac5f5fd7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:19.303088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 493/983] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 06:18:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:19.308941 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc5eb7-2be7-468a-a990-3f0e88465608 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:19.419413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc5eb7-2be7-468a-a990-3f0e88465608 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:19.456119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31959457-e443-452c-8006-e6fc9ca8452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:19.457056 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31959457-e443-452c-8006-e6fc9ca8452c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:19.457503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31959457-e443-452c-8006-e6fc9ca8452c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:19.457967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31959457-e443-452c-8006-e6fc9ca8452c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:19.527151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:19.529633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31959457-e443-452c-8006-e6fc9ca8452c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:19.531770 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31959457-e443-452c-8006-e6fc9ca8452c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:19.638408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 491/984] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:19 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 06:18:19.640142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cb30e-375b-443d-8f69-ca5cac5f5fd7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:19.702044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:19.745557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cb30e-375b-443d-8f69-ca5cac5f5fd7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:19.781208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df3229-4fa9-41b8-94a9-a41f80ec0d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:19.782040 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df3229-4fa9-41b8-94a9-a41f80ec0d6c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:19.782331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df3229-4fa9-41b8-94a9-a41f80ec0d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:19.782613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df3229-4fa9-41b8-94a9-a41f80ec0d6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:19.793481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df3229-4fa9-41b8-94a9-a41f80ec0d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:19.795181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df3229-4fa9-41b8-94a9-a41f80ec0d6c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-06df3229-4fa9-41b8-94a9-a41f80ec0d6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:19.814163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:19.815066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:19.815973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:19.815973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:19.815973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:19.815973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:19.820015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 494/985] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:19.821367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31959457-e443-452c-8006-e6fc9ca8452c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:19.924692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:19.928610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31959457-e443-452c-8006-e6fc9ca8452c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:19.942920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a15243-a43b-44b8-a80f-9b85f2efb582 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:19.943283 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a15243-a43b-44b8-a80f-9b85f2efb582 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:19.943642 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a15243-a43b-44b8-a80f-9b85f2efb582 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:19.945141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 495/986] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:19.950353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4dd1b-eae8-4d54-a172-c76e19926cc9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:19.950644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4dd1b-eae8-4d54-a172-c76e19926cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:19.950925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4dd1b-eae8-4d54-a172-c76e19926cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:19.967853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f5e4dd1b-eae8-4d54-a172-c76e19926cc9 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:19.968233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ff78c-0b85-41a1-9d77-090cbdf2fb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:19.969011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f5e4dd1b-eae8-4d54-a172-c76e19926cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:19.969316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ff78c-0b85-41a1-9d77-090cbdf2fb37 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:19.969790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ff78c-0b85-41a1-9d77-090cbdf2fb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:19.970276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ff78c-0b85-41a1-9d77-090cbdf2fb37 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:19.971306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ff78c-0b85-41a1-9d77-090cbdf2fb37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:19.971930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ff78c-0b85-41a1-9d77-090cbdf2fb37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:19.973298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ff78c-0b85-41a1-9d77-090cbdf2fb37 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:19.992361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 492/987] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:19.993969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ff78c-0b85-41a1-9d77-090cbdf2fb37 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:20.061984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 496/988] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:20.075961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4dd1b-eae8-4d54-a172-c76e19926cc9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:20.108563 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ff78c-0b85-41a1-9d77-090cbdf2fb37 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:20.126016 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:20.145556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe53ed1-3908-4a8d-a973-5f269f50917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:20.146406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe53ed1-3908-4a8d-a973-5f269f50917f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:20.146691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe53ed1-3908-4a8d-a973-5f269f50917f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:20.147244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe53ed1-3908-4a8d-a973-5f269f50917f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:20.154989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe53ed1-3908-4a8d-a973-5f269f50917f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:20.156437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe53ed1-3908-4a8d-a973-5f269f50917f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-9fe53ed1-3908-4a8d-a973-5f269f50917f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:18:20.171525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:20.172443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:20.173328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:18:20.173328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:18:20.173328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:18:20.173328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:18:20.173328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:18:20.173328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:18:20.173328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:20.173328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:20.173328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 493/989] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:20.178773 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe53ed1-3908-4a8d-a973-5f269f50917f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:20.194521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4dd1b-eae8-4d54-a172-c76e19926cc9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:20.202520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:20.231372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2507af-a801-493b-814b-9de5c88f0b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:20.232106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2507af-a801-493b-814b-9de5c88f0b92 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:20.232390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2507af-a801-493b-814b-9de5c88f0b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:20.232728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2507af-a801-493b-814b-9de5c88f0b92 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:20.233386 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2507af-a801-493b-814b-9de5c88f0b92 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:20.234844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2507af-a801-493b-814b-9de5c88f0b92 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7f2507af-a801-493b-814b-9de5c88f0b92 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:20.247208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:20.248199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:20.249046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 497/990] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 06:18:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:20.254354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2507af-a801-493b-814b-9de5c88f0b92 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:20.291432 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe53ed1-3908-4a8d-a973-5f269f50917f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:20.327242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4540a2-b32f-40b1-9db5-de47967566c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:20.327902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4540a2-b32f-40b1-9db5-de47967566c7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:20.328189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4540a2-b32f-40b1-9db5-de47967566c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:20.328470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4540a2-b32f-40b1-9db5-de47967566c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:20.329264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4540a2-b32f-40b1-9db5-de47967566c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:20.329973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4540a2-b32f-40b1-9db5-de47967566c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:20.331035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4540a2-b32f-40b1-9db5-de47967566c7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:20.343851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 494/991] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:18:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:20.349790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4540a2-b32f-40b1-9db5-de47967566c7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:20.361342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2507af-a801-493b-814b-9de5c88f0b92 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:20.363848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:20.399189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9ba06-c4d6-433a-a690-7a8c30b33636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:20.400115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9ba06-c4d6-433a-a690-7a8c30b33636 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:20.400582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9ba06-c4d6-433a-a690-7a8c30b33636 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:20.401100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9ba06-c4d6-433a-a690-7a8c30b33636 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:20.402720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9ba06-c4d6-433a-a690-7a8c30b33636 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:20.403210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9ba06-c4d6-433a-a690-7a8c30b33636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:20.404755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9ba06-c4d6-433a-a690-7a8c30b33636 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:20.410170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 498/992] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 06:18:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:20.415792 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9ba06-c4d6-433a-a690-7a8c30b33636 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:20.438771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:20.460067 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4540a2-b32f-40b1-9db5-de47967566c7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:20.494491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca961cd-b587-4921-a91b-01f746679a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:20.496237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca961cd-b587-4921-a91b-01f746679a0a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:20.496545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca961cd-b587-4921-a91b-01f746679a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:20.496851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca961cd-b587-4921-a91b-01f746679a0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:20.497671 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca961cd-b587-4921-a91b-01f746679a0a None admin] RBAC: Adding query filter params (role.id=d726d771b3d34d378b43824cbfec19c9, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=e0ff633636ec403780dd4bd21e0fa65a) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:20.498008 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca961cd-b587-4921-a91b-01f746679a0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:20.499437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca961cd-b587-4921-a91b-01f746679a0a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:20.505485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 495/993] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:18:20 2022] GET /identity/v3/role_assignments?user.id=e0ff633636ec403780dd4bd21e0fa65a&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=d726d771b3d34d378b43824cbfec19c9 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:20.541239 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9ba06-c4d6-433a-a690-7a8c30b33636 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:20.579985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049441e1-3dd9-440c-bf8d-bff9c93b032b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:20.580659 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049441e1-3dd9-440c-bf8d-bff9c93b032b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:20.580941 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049441e1-3dd9-440c-bf8d-bff9c93b032b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:20.581214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049441e1-3dd9-440c-bf8d-bff9c93b032b None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:20.581932 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049441e1-3dd9-440c-bf8d-bff9c93b032b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:20.582261 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049441e1-3dd9-440c-bf8d-bff9c93b032b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:20.583858 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049441e1-3dd9-440c-bf8d-bff9c93b032b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:20.595955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 499/994] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 06:18:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:20.597075 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca961cd-b587-4921-a91b-01f746679a0a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:20.702807 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca961cd-b587-4921-a91b-01f746679a0a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:20.737855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e309c8-dc6b-4697-b40c-1d982f00c8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:20.738557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e309c8-dc6b-4697-b40c-1d982f00c8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:20.738842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e309c8-dc6b-4697-b40c-1d982f00c8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:20.739122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e309c8-dc6b-4697-b40c-1d982f00c8b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:20.740077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e309c8-dc6b-4697-b40c-1d982f00c8b8 None admin] RBAC: Adding query filter params (interface=admin, service_id=d7bf3c276c5a45da9886b4c0f3ea49b7, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:20.740397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e309c8-dc6b-4697-b40c-1d982f00c8b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:20.741897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e309c8-dc6b-4697-b40c-1d982f00c8b8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:20.748500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 496/995] 10.222.0.42 () {56 vars in 1249 bytes} [Thu Jul 21 06:18:20 2022] GET /identity/v3/endpoints?service_id=d7bf3c276c5a45da9886b4c0f3ea49b7&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:20.753339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049441e1-3dd9-440c-bf8d-bff9c93b032b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:20.787769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:20.860919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049441e1-3dd9-440c-bf8d-bff9c93b032b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:20.894515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cfc3b-6313-476c-a568-18cf67c1d68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:20.895192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cfc3b-6313-476c-a568-18cf67c1d68b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:20.895478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cfc3b-6313-476c-a568-18cf67c1d68b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:20.895760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cfc3b-6313-476c-a568-18cf67c1d68b None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:20.896449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cfc3b-6313-476c-a568-18cf67c1d68b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:20.898110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cfc3b-6313-476c-a568-18cf67c1d68b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:20.905141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 500/996] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 06:18:20 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:18:21.085339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533576-f0ed-445a-84c9-29f690ead443 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:21.085690 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533576-f0ed-445a-84c9-29f690ead443 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:21.085919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533576-f0ed-445a-84c9-29f690ead443 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:21.087474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 497/997] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:21.093188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a2838-d244-498f-8eb4-37795508d927 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:21.093631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a2838-d244-498f-8eb4-37795508d927 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:21.093823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a2838-d244-498f-8eb4-37795508d927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:21.110813 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-362a2838-d244-498f-8eb4-37795508d927 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:21.112005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-362a2838-d244-498f-8eb4-37795508d927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:21.180093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 501/998] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:21.204605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2533576-f0ed-445a-84c9-29f690ead443 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:21.217110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:21.312328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2533576-f0ed-445a-84c9-29f690ead443 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:21.347424 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549de44-1e1e-48b1-9157-87e227bd47f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:21.348224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549de44-1e1e-48b1-9157-87e227bd47f5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:21.348514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549de44-1e1e-48b1-9157-87e227bd47f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:21.348796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549de44-1e1e-48b1-9157-87e227bd47f5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:21.349452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b549de44-1e1e-48b1-9157-87e227bd47f5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:21.350866 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b549de44-1e1e-48b1-9157-87e227bd47f5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b549de44-1e1e-48b1-9157-87e227bd47f5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:21.362819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:21.363831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 06:18:21.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:21.366111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 498/999] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:21.369896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a2838-d244-498f-8eb4-37795508d927 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:21.399060 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:21.475772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a2838-d244-498f-8eb4-37795508d927 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:21.510234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4913997-6801-4e54-8af3-fa441e8cc4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:21.511763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4913997-6801-4e54-8af3-fa441e8cc4ae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:21.511889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4913997-6801-4e54-8af3-fa441e8cc4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:21.512149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4913997-6801-4e54-8af3-fa441e8cc4ae None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:21.512861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4913997-6801-4e54-8af3-fa441e8cc4ae None admin] RBAC: Adding query filter params (name=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:21.513173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4913997-6801-4e54-8af3-fa441e8cc4ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:21.514569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4913997-6801-4e54-8af3-fa441e8cc4ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:21.524117 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 502/1000] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:18:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:21.525176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549de44-1e1e-48b1-9157-87e227bd47f5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:21.598748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:21.630274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549de44-1e1e-48b1-9157-87e227bd47f5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:21.667461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3971a-7a2d-4be0-b5de-57d3b34c65d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:21.668136 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3971a-7a2d-4be0-b5de-57d3b34c65d9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:21.668424 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3971a-7a2d-4be0-b5de-57d3b34c65d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:21.668704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3971a-7a2d-4be0-b5de-57d3b34c65d9 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:21.669420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3971a-7a2d-4be0-b5de-57d3b34c65d9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:21.669732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3971a-7a2d-4be0-b5de-57d3b34c65d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:21.671155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3971a-7a2d-4be0-b5de-57d3b34c65d9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:21.676542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 499/1001] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 06:18:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:21.682208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4913997-6801-4e54-8af3-fa441e8cc4ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:21.775322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:21.786303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4913997-6801-4e54-8af3-fa441e8cc4ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:21.820991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea3418-65d0-47b1-af15-4e789233e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:21.821655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea3418-65d0-47b1-af15-4e789233e459 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:21.821965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea3418-65d0-47b1-af15-4e789233e459 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:21.822248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea3418-65d0-47b1-af15-4e789233e459 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:21.823030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea3418-65d0-47b1-af15-4e789233e459 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:21.824570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea3418-65d0-47b1-af15-4e789233e459 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:21.845847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 503/1002] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 06:18:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:18:22.622776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e579e36-e327-4a93-8b75-39dafcf9b05b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:22.623327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e579e36-e327-4a93-8b75-39dafcf9b05b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:22.623327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e579e36-e327-4a93-8b75-39dafcf9b05b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:22.624728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 500/1003] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:22.630033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48db71-f1ed-4a81-a77e-036f1eeeeab7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:22.630434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48db71-f1ed-4a81-a77e-036f1eeeeab7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:22.630595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48db71-f1ed-4a81-a77e-036f1eeeeab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:22.648351 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4d48db71-f1ed-4a81-a77e-036f1eeeeab7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:22.649518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4d48db71-f1ed-4a81-a77e-036f1eeeeab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:22.724489 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 504/1004] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:22 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:22.746228 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e579e36-e327-4a93-8b75-39dafcf9b05b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:22.848580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e579e36-e327-4a93-8b75-39dafcf9b05b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:22.882097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e10e0-01a8-4266-a4af-310002b2d9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:22.882816 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e10e0-01a8-4266-a4af-310002b2d9cf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:22.883094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e10e0-01a8-4266-a4af-310002b2d9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:22.883372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e10e0-01a8-4266-a4af-310002b2d9cf None admin] PATH_INFO: `/v3/services/image` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:22.884021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e10e0-01a8-4266-a4af-310002b2d9cf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:22.885374 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e10e0-01a8-4266-a4af-310002b2d9cf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:22.888697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-699e10e0-01a8-4266-a4af-310002b2d9cf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:22.899771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:22.900829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 06:18:22.902221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:22.906110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 501/1005] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Jul 21 06:18:22 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:22.907596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48db71-f1ed-4a81-a77e-036f1eeeeab7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:22.977730 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:22.981563 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90df10-e793-4859-872a-35e8400078b3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:22.981869 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90df10-e793-4859-872a-35e8400078b3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:22.982394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90df10-e793-4859-872a-35e8400078b3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:22.983896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 502/1006] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:18:22.989272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5406a-21d2-443a-9472-1f892723f707 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:22.989494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5406a-21d2-443a-9472-1f892723f707 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:22.989772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5406a-21d2-443a-9472-1f892723f707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:23.006147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-eda5406a-21d2-443a-9472-1f892723f707 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:23.007319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-eda5406a-21d2-443a-9472-1f892723f707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:23.011420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48db71-f1ed-4a81-a77e-036f1eeeeab7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:23.052768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55889e-f8c4-45cf-b7d9-04f328cb26cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:23.053465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55889e-f8c4-45cf-b7d9-04f328cb26cc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:23.053752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55889e-f8c4-45cf-b7d9-04f328cb26cc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:23.054077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55889e-f8c4-45cf-b7d9-04f328cb26cc None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:23.054836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55889e-f8c4-45cf-b7d9-04f328cb26cc None admin] RBAC: Adding query filter params (name=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:23.055130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55889e-f8c4-45cf-b7d9-04f328cb26cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:23.056893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55889e-f8c4-45cf-b7d9-04f328cb26cc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:23.063181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 505/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:23.069580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55889e-f8c4-45cf-b7d9-04f328cb26cc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:23.100163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 503/1008] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:22 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:23.124051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5406a-21d2-443a-9472-1f892723f707 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:23.185888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55889e-f8c4-45cf-b7d9-04f328cb26cc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:23.222923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0b650-7537-46ed-a5b1-eed05779a22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:23.223875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0b650-7537-46ed-a5b1-eed05779a22c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:23.224182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0b650-7537-46ed-a5b1-eed05779a22c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:23.224428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0b650-7537-46ed-a5b1-eed05779a22c None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:23.225137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0b650-7537-46ed-a5b1-eed05779a22c None admin] RBAC: Adding query filter params (type=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:23.225442 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0b650-7537-46ed-a5b1-eed05779a22c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:23.226838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0b650-7537-46ed-a5b1-eed05779a22c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:23.228934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:23.232564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 506/1009] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:23 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:23.232723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5406a-21d2-443a-9472-1f892723f707 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:23.267149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:23.274444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea352b49-149a-473b-b422-88353a3ae39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:23.275268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea352b49-149a-473b-b422-88353a3ae39d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:23.275555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea352b49-149a-473b-b422-88353a3ae39d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:23.275831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea352b49-149a-473b-b422-88353a3ae39d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:23.277294 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea352b49-149a-473b-b422-88353a3ae39d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:23.278752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea352b49-149a-473b-b422-88353a3ae39d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ea352b49-149a-473b-b422-88353a3ae39d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:23.292611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:23.293731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 06:18:23.294621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:23.298315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 504/1010] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 06:18:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:23.300787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0b650-7537-46ed-a5b1-eed05779a22c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:23.414887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0b650-7537-46ed-a5b1-eed05779a22c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:23.452188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894bb98-2c12-4b0f-a594-947cc00eef8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:23.452950 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894bb98-2c12-4b0f-a594-947cc00eef8a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:23.453333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894bb98-2c12-4b0f-a594-947cc00eef8a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:23.453720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894bb98-2c12-4b0f-a594-947cc00eef8a None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:23.454573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894bb98-2c12-4b0f-a594-947cc00eef8a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:23.454990 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894bb98-2c12-4b0f-a594-947cc00eef8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:23.456518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894bb98-2c12-4b0f-a594-947cc00eef8a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:23.466014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 507/1011] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 06:18:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:23.467637 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea352b49-149a-473b-b422-88353a3ae39d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:23.571819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea352b49-149a-473b-b422-88353a3ae39d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:23.606574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7867b-5a16-46b6-b372-a294a3b016e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:23.607388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7867b-5a16-46b6-b372-a294a3b016e0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:23.607776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7867b-5a16-46b6-b372-a294a3b016e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:23.608161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7867b-5a16-46b6-b372-a294a3b016e0 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:23.608977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7867b-5a16-46b6-b372-a294a3b016e0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:23.609388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7867b-5a16-46b6-b372-a294a3b016e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:23.610868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7867b-5a16-46b6-b372-a294a3b016e0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:23.621028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 505/1012] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 06:18:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:23.622081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894bb98-2c12-4b0f-a594-947cc00eef8a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:23.687378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:23.726534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894bb98-2c12-4b0f-a594-947cc00eef8a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:23.761975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfbe76-73ed-4dd8-9c76-b8e32b187b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:23.762648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfbe76-73ed-4dd8-9c76-b8e32b187b82 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:23.762934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfbe76-73ed-4dd8-9c76-b8e32b187b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:23.763216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfbe76-73ed-4dd8-9c76-b8e32b187b82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:23.763998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfbe76-73ed-4dd8-9c76-b8e32b187b82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:23.765411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfbe76-73ed-4dd8-9c76-b8e32b187b82 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:23.782713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 508/1013] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 06:18:23 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:18:24.001595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc9e21-0133-4ee4-b2d9-2ae15285bee9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:24.002106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc9e21-0133-4ee4-b2d9-2ae15285bee9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:24.002106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc9e21-0133-4ee4-b2d9-2ae15285bee9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:24.003632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 506/1014] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:24.008627 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39866dd4-6716-4da4-bff6-6589ab07e520 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:24.008931 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39866dd4-6716-4da4-bff6-6589ab07e520 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:24.009213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39866dd4-6716-4da4-bff6-6589ab07e520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:24.027646 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-39866dd4-6716-4da4-bff6-6589ab07e520 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:24.028834 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-39866dd4-6716-4da4-bff6-6589ab07e520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:24.104237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 509/1015] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:24 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:24.127511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc9e21-0133-4ee4-b2d9-2ae15285bee9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:24.190213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:24.234439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc9e21-0133-4ee4-b2d9-2ae15285bee9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:24.269453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104530a4-ca1c-44a7-842b-f0e8e827428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:24.270229 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104530a4-ca1c-44a7-842b-f0e8e827428d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:24.270518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104530a4-ca1c-44a7-842b-f0e8e827428d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:24.270795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104530a4-ca1c-44a7-842b-f0e8e827428d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:24.282210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104530a4-ca1c-44a7-842b-f0e8e827428d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-104530a4-ca1c-44a7-842b-f0e8e827428d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:24.290422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:24.291343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:24.292208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:24.293441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 507/1016] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 06:18:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:24.297585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39866dd4-6716-4da4-bff6-6589ab07e520 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:24.352464 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:24.406967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39866dd4-6716-4da4-bff6-6589ab07e520 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:24.441545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fb3cd-6d87-4a89-9f41-d622189031e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:24.442246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fb3cd-6d87-4a89-9f41-d622189031e9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:24.442532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fb3cd-6d87-4a89-9f41-d622189031e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:24.442811 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fb3cd-6d87-4a89-9f41-d622189031e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:24.443565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fb3cd-6d87-4a89-9f41-d622189031e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:24.443876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fb3cd-6d87-4a89-9f41-d622189031e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:24.445242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fb3cd-6d87-4a89-9f41-d622189031e9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:24.458827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 510/1017] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 06:18:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:24.461058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104530a4-ca1c-44a7-842b-f0e8e827428d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:24.563760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104530a4-ca1c-44a7-842b-f0e8e827428d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:24.581055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:24.599082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14976645-8cd2-43b2-893e-953fecf04126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:24.599883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976645-8cd2-43b2-893e-953fecf04126 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:24.600170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976645-8cd2-43b2-893e-953fecf04126 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:24.600451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976645-8cd2-43b2-893e-953fecf04126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:24.611180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14976645-8cd2-43b2-893e-953fecf04126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:24.612612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14976645-8cd2-43b2-893e-953fecf04126 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-14976645-8cd2-43b2-893e-953fecf04126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:24.629880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:24.630881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:24.631771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:24.631771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:24.631771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:24.631771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:24.632489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 508/1018] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:24.637474 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fb3cd-6d87-4a89-9f41-d622189031e9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:24.740344 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fb3cd-6d87-4a89-9f41-d622189031e9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:24.753686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:24.776044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460c2bd-a80f-4367-b071-cc8bf19ba9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:24.776830 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460c2bd-a80f-4367-b071-cc8bf19ba9ed None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:24.777023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460c2bd-a80f-4367-b071-cc8bf19ba9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:24.777302 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460c2bd-a80f-4367-b071-cc8bf19ba9ed None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:24.778119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460c2bd-a80f-4367-b071-cc8bf19ba9ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:24.778438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460c2bd-a80f-4367-b071-cc8bf19ba9ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:24.779854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460c2bd-a80f-4367-b071-cc8bf19ba9ed None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:24.797870 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 511/1019] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:24.799092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14976645-8cd2-43b2-893e-953fecf04126 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:24.902819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14976645-8cd2-43b2-893e-953fecf04126 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:24.938251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2906771-35a8-490d-b7d2-2e3f7756126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:24.939055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906771-35a8-490d-b7d2-2e3f7756126e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:24.939337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906771-35a8-490d-b7d2-2e3f7756126e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:24.939615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906771-35a8-490d-b7d2-2e3f7756126e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:24.947244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2906771-35a8-490d-b7d2-2e3f7756126e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:24.949256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2906771-35a8-490d-b7d2-2e3f7756126e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f2906771-35a8-490d-b7d2-2e3f7756126e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:18:24.964093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:24.965201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:24.966201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:24.966201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:24.966201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:18:24.966201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:18:24.966201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:18:24.966201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:18:24.966201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:18:24.966201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:18:24.966201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:18:24.966201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:24.966201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 509/1020] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 06:18:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:24.971703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460c2bd-a80f-4367-b071-cc8bf19ba9ed None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:25.084200 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460c2bd-a80f-4367-b071-cc8bf19ba9ed None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:25.121574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50bef5-ec3b-47e9-ae28-6528708127a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:25.122280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50bef5-ec3b-47e9-ae28-6528708127a1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:25.122568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50bef5-ec3b-47e9-ae28-6528708127a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:25.123229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50bef5-ec3b-47e9-ae28-6528708127a1 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:25.124800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50bef5-ec3b-47e9-ae28-6528708127a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:25.125019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50bef5-ec3b-47e9-ae28-6528708127a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:25.126505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50bef5-ec3b-47e9-ae28-6528708127a1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:25.157126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 512/1021] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 06:18:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:25.161364 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2906771-35a8-490d-b7d2-2e3f7756126e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:25.200307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:25.274840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2906771-35a8-490d-b7d2-2e3f7756126e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:25.310058 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2113bff-9ef1-4f99-8f29-3525f8de8cbb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:25.310385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc32b14-0b6a-4ee5-83ba-550fd3c84658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:25.310511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2113bff-9ef1-4f99-8f29-3525f8de8cbb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:25.310699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2113bff-9ef1-4f99-8f29-3525f8de8cbb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:25.310954 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc32b14-0b6a-4ee5-83ba-550fd3c84658 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:25.311237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc32b14-0b6a-4ee5-83ba-550fd3c84658 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:25.311509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc32b14-0b6a-4ee5-83ba-550fd3c84658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:25.312278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 513/1022] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:25.317064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7a781-9a05-4290-bae8-3ce7e951087b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:25.317350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7a781-9a05-4290-bae8-3ce7e951087b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:25.317625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7a781-9a05-4290-bae8-3ce7e951087b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:25.334292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-80b7a781-9a05-4290-bae8-3ce7e951087b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:25.335575 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-80b7a781-9a05-4290-bae8-3ce7e951087b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:25.363897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:25.384692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc32b14-0b6a-4ee5-83ba-550fd3c84658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:25.387194 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc32b14-0b6a-4ee5-83ba-550fd3c84658 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:25.416025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 514/1023] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:25 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:25.420558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:25.439534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7a781-9a05-4290-bae8-3ce7e951087b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:25.494317 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 510/1024] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:25 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:18:25.499974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc32b14-0b6a-4ee5-83ba-550fd3c84658 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:25.554334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7a781-9a05-4290-bae8-3ce7e951087b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:25.590745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bf12d-6dc3-4ff6-a7da-af47e56bd09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:25.591417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bf12d-6dc3-4ff6-a7da-af47e56bd09d None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:25.591708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bf12d-6dc3-4ff6-a7da-af47e56bd09d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:25.591988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bf12d-6dc3-4ff6-a7da-af47e56bd09d None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:25.592711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bf12d-6dc3-4ff6-a7da-af47e56bd09d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:25.594119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bf12d-6dc3-4ff6-a7da-af47e56bd09d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:25.611576 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 515/1025] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:18:25 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:18:25.617032 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc32b14-0b6a-4ee5-83ba-550fd3c84658 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:25.651020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:25.667798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788efa6-3280-4fd7-a966-bfcfc5320482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:25.668534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788efa6-3280-4fd7-a966-bfcfc5320482 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:25.668820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788efa6-3280-4fd7-a966-bfcfc5320482 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:25.669098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788efa6-3280-4fd7-a966-bfcfc5320482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:25.681147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788efa6-3280-4fd7-a966-bfcfc5320482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:25.682743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788efa6-3280-4fd7-a966-bfcfc5320482 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1788efa6-3280-4fd7-a966-bfcfc5320482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:25.702826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:25.703777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:25.709709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:25.709709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:25.709709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:25.709709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:25.709709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 511/1026] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:25.711318 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bf12d-6dc3-4ff6-a7da-af47e56bd09d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:25.736485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:25.828596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bf12d-6dc3-4ff6-a7da-af47e56bd09d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:25.864591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04803c86-b9e7-4471-b709-1469bef57557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:25.865558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04803c86-b9e7-4471-b709-1469bef57557 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:25.866099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04803c86-b9e7-4471-b709-1469bef57557 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:25.866580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04803c86-b9e7-4471-b709-1469bef57557 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:25.867492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04803c86-b9e7-4471-b709-1469bef57557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:25.867984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04803c86-b9e7-4471-b709-1469bef57557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:25.869535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04803c86-b9e7-4471-b709-1469bef57557 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:25.883237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 516/1027] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:25.889196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788efa6-3280-4fd7-a966-bfcfc5320482 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:25.931049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786395d-3fb8-4027-ba74-08adc29d6a3b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:25.931342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786395d-3fb8-4027-ba74-08adc29d6a3b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:25.931532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786395d-3fb8-4027-ba74-08adc29d6a3b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:25.933069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 517/1028] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:25.937961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04abb07-b280-401d-999a-062f84250e8a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:25.938253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04abb07-b280-401d-999a-062f84250e8a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:25.938539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04abb07-b280-401d-999a-062f84250e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:25.958340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e04abb07-b280-401d-999a-062f84250e8a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:25.959534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e04abb07-b280-401d-999a-062f84250e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:25.997180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788efa6-3280-4fd7-a966-bfcfc5320482 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:26.033402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07a92e-24d2-4f61-91dd-baee9175aaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:26.034231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07a92e-24d2-4f61-91dd-baee9175aaeb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:26.034518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07a92e-24d2-4f61-91dd-baee9175aaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:26.034796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07a92e-24d2-4f61-91dd-baee9175aaeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:26.041809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07a92e-24d2-4f61-91dd-baee9175aaeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:26.046105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 518/1029] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:26.046235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07a92e-24d2-4f61-91dd-baee9175aaeb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:26.059470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04abb07-b280-401d-999a-062f84250e8a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3d07a92e-24d2-4f61-91dd-baee9175aaeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:18:26.062373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:26.063221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:26.064102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:18:26.064102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:18:26.064102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:18:26.064102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:18:26.064102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:18:26.064102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:18:26.064102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:26.064102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:26.064102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 512/1030] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:26.069282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07a92e-24d2-4f61-91dd-baee9175aaeb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:26.074848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:26.115785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:26.179899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04abb07-b280-401d-999a-062f84250e8a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:26.181575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07a92e-24d2-4f61-91dd-baee9175aaeb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:26.216627 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c072c3-e614-4937-93a6-2760782291bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:26.217431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c072c3-e614-4937-93a6-2760782291bb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:26.217723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c072c3-e614-4937-93a6-2760782291bb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:26.218030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c072c3-e614-4937-93a6-2760782291bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:26.219206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c072c3-e614-4937-93a6-2760782291bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:26.220373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dcac5-1de5-43f2-a93c-5d55701d2b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:26.220653 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c072c3-e614-4937-93a6-2760782291bb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:26.221099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dcac5-1de5-43f2-a93c-5d55701d2b59 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:26.221380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dcac5-1de5-43f2-a93c-5d55701d2b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:26.221673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dcac5-1de5-43f2-a93c-5d55701d2b59 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:26.226127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dcac5-1de5-43f2-a93c-5d55701d2b59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:26.226263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 519/1031] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:26.226359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dcac5-1de5-43f2-a93c-5d55701d2b59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:26.226359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dcac5-1de5-43f2-a93c-5d55701d2b59 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:26.228101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c072c3-e614-4937-93a6-2760782291bb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:26.237438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 513/1032] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:18:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:26.243611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dcac5-1de5-43f2-a93c-5d55701d2b59 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:26.314869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:26.338090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c072c3-e614-4937-93a6-2760782291bb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:26.344777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:26.350176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dcac5-1de5-43f2-a93c-5d55701d2b59 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:26.376404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20ca7c-1ad4-4954-8be6-8d1b3dcddf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:26.377069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20ca7c-1ad4-4954-8be6-8d1b3dcddf35 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:26.377349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20ca7c-1ad4-4954-8be6-8d1b3dcddf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:26.377622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20ca7c-1ad4-4954-8be6-8d1b3dcddf35 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:26.378447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20ca7c-1ad4-4954-8be6-8d1b3dcddf35 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:26.379879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20ca7c-1ad4-4954-8be6-8d1b3dcddf35 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:26.387225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd22e0d-8e07-4424-836c-ec72f48f37b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:26.388123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd22e0d-8e07-4424-836c-ec72f48f37b0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:26.388605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd22e0d-8e07-4424-836c-ec72f48f37b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:26.388884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd22e0d-8e07-4424-836c-ec72f48f37b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:26.389724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd22e0d-8e07-4424-836c-ec72f48f37b0 None admin] RBAC: Adding query filter params (role.id=d726d771b3d34d378b43824cbfec19c9, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=058f099e6f7545a1b2734dbe5afd2027) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:26.390074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd22e0d-8e07-4424-836c-ec72f48f37b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:26.391516 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd22e0d-8e07-4424-836c-ec72f48f37b0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:26.397609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 514/1033] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:18:26 2022] GET /identity/v3/role_assignments?user.id=058f099e6f7545a1b2734dbe5afd2027&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=d726d771b3d34d378b43824cbfec19c9 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:26.407355 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 520/1034] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:18:26 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:18:27.685634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198c3d3-d502-447e-a7f0-3aa59d898c42 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:27.686200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198c3d3-d502-447e-a7f0-3aa59d898c42 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:27.686280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198c3d3-d502-447e-a7f0-3aa59d898c42 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:27.687638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 515/1035] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:27.692767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed4d41-4615-42d9-a32b-034365cf2206 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:27.693143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed4d41-4615-42d9-a32b-034365cf2206 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:27.693327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed4d41-4615-42d9-a32b-034365cf2206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:27.710309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-68ed4d41-4615-42d9-a32b-034365cf2206 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:27.711481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-68ed4d41-4615-42d9-a32b-034365cf2206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:27.781198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 521/1036] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:27.803398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198c3d3-d502-447e-a7f0-3aa59d898c42 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:27.907473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198c3d3-d502-447e-a7f0-3aa59d898c42 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:27.944759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce78a3-c1dc-4248-bb98-fb2a8ad3ee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:27.945473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce78a3-c1dc-4248-bb98-fb2a8ad3ee8a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:27.945759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce78a3-c1dc-4248-bb98-fb2a8ad3ee8a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:27.946068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce78a3-c1dc-4248-bb98-fb2a8ad3ee8a None admin] PATH_INFO: `/v3/services/image` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:27.946724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce78a3-c1dc-4248-bb98-fb2a8ad3ee8a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:27.948588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce78a3-c1dc-4248-bb98-fb2a8ad3ee8a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-38ce78a3-c1dc-4248-bb98-fb2a8ad3ee8a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:27.960629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:27.961632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 06:18:27.962562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:27.966275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 516/1037] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Jul 21 06:18:27 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:27.968327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed4d41-4615-42d9-a32b-034365cf2206 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:28.028950 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:28.080784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed4d41-4615-42d9-a32b-034365cf2206 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:28.114773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be2164-0d74-4fe0-b780-82b569ff85cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:28.115445 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be2164-0d74-4fe0-b780-82b569ff85cf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.115730 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be2164-0d74-4fe0-b780-82b569ff85cf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.116006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be2164-0d74-4fe0-b780-82b569ff85cf None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.116720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be2164-0d74-4fe0-b780-82b569ff85cf None admin] RBAC: Adding query filter params (name=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:28.117028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be2164-0d74-4fe0-b780-82b569ff85cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:28.118423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be2164-0d74-4fe0-b780-82b569ff85cf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:28.124303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 522/1038] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:28.131993 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce78a3-c1dc-4248-bb98-fb2a8ad3ee8a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:28.162352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:28.237331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce78a3-c1dc-4248-bb98-fb2a8ad3ee8a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:28.270896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f6aec-50bc-4f38-9d49-72df6adc4c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:28.271568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f6aec-50bc-4f38-9d49-72df6adc4c4a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.272117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f6aec-50bc-4f38-9d49-72df6adc4c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.272398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f6aec-50bc-4f38-9d49-72df6adc4c4a None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.273110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f6aec-50bc-4f38-9d49-72df6adc4c4a None admin] RBAC: Adding query filter params (type=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:28.273419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f6aec-50bc-4f38-9d49-72df6adc4c4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:28.274815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f6aec-50bc-4f38-9d49-72df6adc4c4a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:28.285430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 517/1039] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:28.286835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be2164-0d74-4fe0-b780-82b569ff85cf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:28.391003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be2164-0d74-4fe0-b780-82b569ff85cf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:28.415276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:28.427774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd5888-ab2d-4cea-b8f2-7acb13e6f5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:28.428441 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd5888-ab2d-4cea-b8f2-7acb13e6f5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.428726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd5888-ab2d-4cea-b8f2-7acb13e6f5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.429004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd5888-ab2d-4cea-b8f2-7acb13e6f5b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.429803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd5888-ab2d-4cea-b8f2-7acb13e6f5b1 None admin] RBAC: Adding query filter params (interface=public, service_id=9cd4ac4dde7e4bb19b0a953a91bfe9fa, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:28.430147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd5888-ab2d-4cea-b8f2-7acb13e6f5b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:28.431515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd5888-ab2d-4cea-b8f2-7acb13e6f5b1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:28.433436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae17d07-a7fe-446f-a08e-9cf4346b64f9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.433732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae17d07-a7fe-446f-a08e-9cf4346b64f9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.434058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae17d07-a7fe-446f-a08e-9cf4346b64f9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.435568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 518/1040] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:18:28.438429 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 523/1041] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 06:18:28 2022] GET /identity/v3/endpoints?service_id=9cd4ac4dde7e4bb19b0a953a91bfe9fa&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:28.440291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b839a-0de0-43f4-83e3-241fe85d1483 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.440583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b839a-0de0-43f4-83e3-241fe85d1483 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.440868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b839a-0de0-43f4-83e3-241fe85d1483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.444077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd5888-ab2d-4cea-b8f2-7acb13e6f5b1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:28.458485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-637b839a-0de0-43f4-83e3-241fe85d1483 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:28.459661 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-637b839a-0de0-43f4-83e3-241fe85d1483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:28.537104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 519/1042] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:28.538372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15839-6c0a-4691-b812-8465b8170450 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.538664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15839-6c0a-4691-b812-8465b8170450 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.538942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15839-6c0a-4691-b812-8465b8170450 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.540385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 520/1043] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:28.545844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de667ed1-4c72-4b5f-a91c-1b0dce0a1c4a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.546157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de667ed1-4c72-4b5f-a91c-1b0dce0a1c4a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.546432 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de667ed1-4c72-4b5f-a91c-1b0dce0a1c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.556971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd5888-ab2d-4cea-b8f2-7acb13e6f5b1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:28.564870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-de667ed1-4c72-4b5f-a91c-1b0dce0a1c4a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:28.566126 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-de667ed1-4c72-4b5f-a91c-1b0dce0a1c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:28.581268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:28.593067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a17b80-d03d-466b-9891-aff0618aea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:28.593743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a17b80-d03d-466b-9891-aff0618aea45 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.594063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a17b80-d03d-466b-9891-aff0618aea45 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.594344 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a17b80-d03d-466b-9891-aff0618aea45 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.595029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a17b80-d03d-466b-9891-aff0618aea45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:28.596388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a17b80-d03d-466b-9891-aff0618aea45 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:28.602856 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 524/1044] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 06:18:28 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:18:28.603633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a17b80-d03d-466b-9891-aff0618aea45 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:28.638768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 521/1045] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:28.663901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de667ed1-4c72-4b5f-a91c-1b0dce0a1c4a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:28.669253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:28.727818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a17b80-d03d-466b-9891-aff0618aea45 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:28.773049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d9cf6-efd0-47ab-af40-e90038baac4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:28.773885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d9cf6-efd0-47ab-af40-e90038baac4c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.774220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d9cf6-efd0-47ab-af40-e90038baac4c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.774401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de667ed1-4c72-4b5f-a91c-1b0dce0a1c4a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:28.774816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d9cf6-efd0-47ab-af40-e90038baac4c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.786846 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d9cf6-efd0-47ab-af40-e90038baac4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ae8d9cf6-efd0-47ab-af40-e90038baac4c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:28.795226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:28.796151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:28.797030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 525/1046] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 06:18:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:28.804048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d9cf6-efd0-47ab-af40-e90038baac4c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:28.818380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bca198-26dd-421d-9076-a95fbc750e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:28.819322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bca198-26dd-421d-9076-a95fbc750e19 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.819725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bca198-26dd-421d-9076-a95fbc750e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.820058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bca198-26dd-421d-9076-a95fbc750e19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.821471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bca198-26dd-421d-9076-a95fbc750e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:28.822930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bca198-26dd-421d-9076-a95fbc750e19 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:28.828205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 522/1047] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:28.831610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bca198-26dd-421d-9076-a95fbc750e19 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:28.851275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:28.910640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d9cf6-efd0-47ab-af40-e90038baac4c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:28.923832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:28.942334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bca198-26dd-421d-9076-a95fbc750e19 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:28.946794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba2e5e-b8f8-4db7-a57b-ac23004defee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:28.947639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba2e5e-b8f8-4db7-a57b-ac23004defee None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.948100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba2e5e-b8f8-4db7-a57b-ac23004defee None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.949125 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba2e5e-b8f8-4db7-a57b-ac23004defee None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.950052 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba2e5e-b8f8-4db7-a57b-ac23004defee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:28.950644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba2e5e-b8f8-4db7-a57b-ac23004defee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:28.952025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba2e5e-b8f8-4db7-a57b-ac23004defee None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:28.970488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 526/1048] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 06:18:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:28.971961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba2e5e-b8f8-4db7-a57b-ac23004defee None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:28.988409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ee774-4c01-4355-8a6a-5174f094955e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:28.989079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee774-4c01-4355-8a6a-5174f094955e None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:28.989377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee774-4c01-4355-8a6a-5174f094955e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:28.989636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee774-4c01-4355-8a6a-5174f094955e None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:28.990434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ee774-4c01-4355-8a6a-5174f094955e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:28.991801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ee774-4c01-4355-8a6a-5174f094955e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:29.016509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 523/1049] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:18:28 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:18:29.100647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba2e5e-b8f8-4db7-a57b-ac23004defee None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:29.138596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd589181-2cc4-4fd4-9ef2-e2c42e364532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:29.139440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd589181-2cc4-4fd4-9ef2-e2c42e364532 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:29.139725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd589181-2cc4-4fd4-9ef2-e2c42e364532 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:29.140001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd589181-2cc4-4fd4-9ef2-e2c42e364532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:29.152725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd589181-2cc4-4fd4-9ef2-e2c42e364532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:29.154230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd589181-2cc4-4fd4-9ef2-e2c42e364532 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-cd589181-2cc4-4fd4-9ef2-e2c42e364532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:29.173021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:29.174025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:29.175221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:29.175221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:29.175221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:29.175221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:29.175221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 527/1050] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:29.180031 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ee774-4c01-4355-8a6a-5174f094955e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:29.270122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:29.298307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ee774-4c01-4355-8a6a-5174f094955e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:29.334127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4666ed-6034-49d5-98c8-aff01e4d9663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:29.334731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4666ed-6034-49d5-98c8-aff01e4d9663 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:29.335012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4666ed-6034-49d5-98c8-aff01e4d9663 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:29.335288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4666ed-6034-49d5-98c8-aff01e4d9663 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:29.336021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4666ed-6034-49d5-98c8-aff01e4d9663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:29.336332 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4666ed-6034-49d5-98c8-aff01e4d9663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:29.337696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4666ed-6034-49d5-98c8-aff01e4d9663 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:29.350242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 524/1051] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:29.356532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd589181-2cc4-4fd4-9ef2-e2c42e364532 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:29.386408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:29.467297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd589181-2cc4-4fd4-9ef2-e2c42e364532 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:29.502691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dca92e-3421-44ce-9b73-8a77da643364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:29.503771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dca92e-3421-44ce-9b73-8a77da643364 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:29.504259 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dca92e-3421-44ce-9b73-8a77da643364 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:29.504717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dca92e-3421-44ce-9b73-8a77da643364 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:29.512581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dca92e-3421-44ce-9b73-8a77da643364 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:29.514245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dca92e-3421-44ce-9b73-8a77da643364 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a3dca92e-3421-44ce-9b73-8a77da643364 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:18:29.528440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:29.529461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:29.530417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:29.530417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:29.530417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:18:29.530417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:18:29.530417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:18:29.530417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:18:29.530417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:18:29.530417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:18:29.530417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:18:29.530417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:29.534006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 528/1052] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 06:18:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:29.536244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4666ed-6034-49d5-98c8-aff01e4d9663 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:29.640737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4666ed-6034-49d5-98c8-aff01e4d9663 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:29.675232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96957f46-48ee-4761-abfd-1a928e7f258e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:29.675910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96957f46-48ee-4761-abfd-1a928e7f258e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:29.676193 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96957f46-48ee-4761-abfd-1a928e7f258e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:29.676478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96957f46-48ee-4761-abfd-1a928e7f258e None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:29.677319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96957f46-48ee-4761-abfd-1a928e7f258e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:29.677632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96957f46-48ee-4761-abfd-1a928e7f258e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:29.679078 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96957f46-48ee-4761-abfd-1a928e7f258e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:29.680788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:29.700480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 525/1053] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 06:18:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:29.705872 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dca92e-3421-44ce-9b73-8a77da643364 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:29.801048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:29.818162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dca92e-3421-44ce-9b73-8a77da643364 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:29.851793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdd247-e951-4149-807f-bbab62d4e8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:29.852592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdd247-e951-4149-807f-bbab62d4e8ae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:29.852871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdd247-e951-4149-807f-bbab62d4e8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:29.853149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdd247-e951-4149-807f-bbab62d4e8ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:29.863691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcdd247-e951-4149-807f-bbab62d4e8ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:29.865140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcdd247-e951-4149-807f-bbab62d4e8ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-abcdd247-e951-4149-807f-bbab62d4e8ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:29.882683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:29.883568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:29.890607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:29.890607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:29.890607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:29.890607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:29.890607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 529/1054] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:29.890937 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96957f46-48ee-4761-abfd-1a928e7f258e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:29.990629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96957f46-48ee-4761-abfd-1a928e7f258e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:30.024761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac8a72-5a57-4f6f-95f8-599739f1d988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:30.025427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac8a72-5a57-4f6f-95f8-599739f1d988 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:30.025712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac8a72-5a57-4f6f-95f8-599739f1d988 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:30.026019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac8a72-5a57-4f6f-95f8-599739f1d988 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:30.026821 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac8a72-5a57-4f6f-95f8-599739f1d988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:30.027129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac8a72-5a57-4f6f-95f8-599739f1d988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:30.028484 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac8a72-5a57-4f6f-95f8-599739f1d988 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:30.044934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 526/1055] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:30.047168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdd247-e951-4149-807f-bbab62d4e8ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:30.155607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdd247-e951-4149-807f-bbab62d4e8ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:30.197232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef8f6-b95a-473d-afc7-c8a6a683a1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:30.198121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef8f6-b95a-473d-afc7-c8a6a683a1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:30.198414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef8f6-b95a-473d-afc7-c8a6a683a1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:30.198692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef8f6-b95a-473d-afc7-c8a6a683a1d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:30.206378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ef8f6-b95a-473d-afc7-c8a6a683a1d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:30.207834 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ef8f6-b95a-473d-afc7-c8a6a683a1d1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:30.209662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e04ef8f6-b95a-473d-afc7-c8a6a683a1d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:18:30.224197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:30.225139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:30.226062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:18:30.226062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:18:30.226062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:18:30.226062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:18:30.226062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:18:30.226062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:18:30.226062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:30.226062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:30.226062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 530/1056] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:30.233220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac8a72-5a57-4f6f-95f8-599739f1d988 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:30.289395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:30.339493 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac8a72-5a57-4f6f-95f8-599739f1d988 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:30.373964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbe0df-9dd0-4e35-9718-4c84d8f89af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:30.374652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbe0df-9dd0-4e35-9718-4c84d8f89af1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:30.374937 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbe0df-9dd0-4e35-9718-4c84d8f89af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:30.375216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbe0df-9dd0-4e35-9718-4c84d8f89af1 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:30.376009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbe0df-9dd0-4e35-9718-4c84d8f89af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:30.376318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbe0df-9dd0-4e35-9718-4c84d8f89af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:30.377738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbe0df-9dd0-4e35-9718-4c84d8f89af1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:30.390511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 527/1057] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:18:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:30.396158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef8f6-b95a-473d-afc7-c8a6a683a1d1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:30.498157 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef8f6-b95a-473d-afc7-c8a6a683a1d1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:30.533423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4474ee9-b51e-4e30-8ad4-107d4f1b984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:30.534344 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4474ee9-b51e-4e30-8ad4-107d4f1b984f None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:30.534628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4474ee9-b51e-4e30-8ad4-107d4f1b984f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:30.534901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4474ee9-b51e-4e30-8ad4-107d4f1b984f None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/058f099e6f7545a1b2734dbe5afd2027/roles/d726d771b3d34d378b43824cbfec19c9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:30.537089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4474ee9-b51e-4e30-8ad4-107d4f1b984f None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=058f099e6f7545a1b2734dbe5afd2027, role_id=d726d771b3d34d378b43824cbfec19c9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:30.538564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4474ee9-b51e-4e30-8ad4-107d4f1b984f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:30.553624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 531/1058] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:18:30 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/058f099e6f7545a1b2734dbe5afd2027/roles/d726d771b3d34d378b43824cbfec19c9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:18:30.669734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b567-5718-4799-be9d-5152eb1de60d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:30.670118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b567-5718-4799-be9d-5152eb1de60d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:30.670332 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b567-5718-4799-be9d-5152eb1de60d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:30.671881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 528/1059] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:30.677374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c263154-ea13-4552-ba58-fde750a27e3c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:30.677652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c263154-ea13-4552-ba58-fde750a27e3c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:30.677957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c263154-ea13-4552-ba58-fde750a27e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:30.695873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5c263154-ea13-4552-ba58-fde750a27e3c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:30.699565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c263154-ea13-4552-ba58-fde750a27e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:30.755380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:30.770418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 532/1060] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:30.794054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b567-5718-4799-be9d-5152eb1de60d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:30.900483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b567-5718-4799-be9d-5152eb1de60d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:30.914917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:30.938144 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79550970-708c-4aaf-9f7f-816bee91d3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:30.938937 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79550970-708c-4aaf-9f7f-816bee91d3ba None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:30.939225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79550970-708c-4aaf-9f7f-816bee91d3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:30.939505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79550970-708c-4aaf-9f7f-816bee91d3ba None admin] PATH_INFO: `/v3/services/image` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:30.940157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79550970-708c-4aaf-9f7f-816bee91d3ba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:30.941509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79550970-708c-4aaf-9f7f-816bee91d3ba None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-79550970-708c-4aaf-9f7f-816bee91d3ba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:30.953066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:30.954119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:30.954976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 529/1061] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Jul 21 06:18:30 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:30.962320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c263154-ea13-4552-ba58-fde750a27e3c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:31.073225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c263154-ea13-4552-ba58-fde750a27e3c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:31.111552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b196a51-fcbc-4504-8300-8f3ef5858480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:31.112226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b196a51-fcbc-4504-8300-8f3ef5858480 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:31.112508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b196a51-fcbc-4504-8300-8f3ef5858480 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:31.112783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b196a51-fcbc-4504-8300-8f3ef5858480 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:31.113494 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b196a51-fcbc-4504-8300-8f3ef5858480 None admin] RBAC: Adding query filter params (name=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:31.113803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b196a51-fcbc-4504-8300-8f3ef5858480 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:31.115206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b196a51-fcbc-4504-8300-8f3ef5858480 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:31.124913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 533/1062] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:31.126155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79550970-708c-4aaf-9f7f-816bee91d3ba None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:31.129100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0265-c151-44e1-bea2-8d799b3ddc91 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:31.129388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0265-c151-44e1-bea2-8d799b3ddc91 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:31.129669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0265-c151-44e1-bea2-8d799b3ddc91 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:31.131203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 534/1063] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:18:31.135930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67644475-3d46-4da8-894e-fb8f8827e90f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:31.136218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67644475-3d46-4da8-894e-fb8f8827e90f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:31.136499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67644475-3d46-4da8-894e-fb8f8827e90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:31.153516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-67644475-3d46-4da8-894e-fb8f8827e90f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:31.154736 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-67644475-3d46-4da8-894e-fb8f8827e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:31.205391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:31.225980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 535/1064] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:31.239692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79550970-708c-4aaf-9f7f-816bee91d3ba None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:31.248899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67644475-3d46-4da8-894e-fb8f8827e90f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:31.277538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959d059-e8e0-471f-a1b1-33035ef46b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:31.278259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959d059-e8e0-471f-a1b1-33035ef46b7a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:31.278548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959d059-e8e0-471f-a1b1-33035ef46b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:31.278830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959d059-e8e0-471f-a1b1-33035ef46b7a None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:31.279539 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959d059-e8e0-471f-a1b1-33035ef46b7a None admin] RBAC: Adding query filter params (type=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:31.279854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959d059-e8e0-471f-a1b1-33035ef46b7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:31.281224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959d059-e8e0-471f-a1b1-33035ef46b7a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:31.286411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 530/1065] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:31.293687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959d059-e8e0-471f-a1b1-33035ef46b7a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:31.336822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:31.361768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67644475-3d46-4da8-894e-fb8f8827e90f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:31.400300 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2517a9-6d45-417d-b82a-2d18b53e78f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:31.400842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959d059-e8e0-471f-a1b1-33035ef46b7a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:31.401101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2517a9-6d45-417d-b82a-2d18b53e78f9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:31.401451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2517a9-6d45-417d-b82a-2d18b53e78f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:31.401730 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2517a9-6d45-417d-b82a-2d18b53e78f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:31.413170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2517a9-6d45-417d-b82a-2d18b53e78f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-cb2517a9-6d45-417d-b82a-2d18b53e78f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:31.421725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:31.422686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:18:31.423521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:31.428058 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 536/1066] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:31.428879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2517a9-6d45-417d-b82a-2d18b53e78f9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:31.439723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed63c9-bcca-4333-b867-1df91d1ae847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:31.440391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed63c9-bcca-4333-b867-1df91d1ae847 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:31.440675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed63c9-bcca-4333-b867-1df91d1ae847 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:31.441166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed63c9-bcca-4333-b867-1df91d1ae847 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:31.441690 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed63c9-bcca-4333-b867-1df91d1ae847 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:31.443088 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed63c9-bcca-4333-b867-1df91d1ae847 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:31.465209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 531/1067] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 06:18:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:18:31.488942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:31.555476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2517a9-6d45-417d-b82a-2d18b53e78f9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:31.602842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c35f37-1bbf-4adc-959d-7e4b0c2d1dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:31.603438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c35f37-1bbf-4adc-959d-7e4b0c2d1dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:31.603722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c35f37-1bbf-4adc-959d-7e4b0c2d1dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:31.603999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c35f37-1bbf-4adc-959d-7e4b0c2d1dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:31.605478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c35f37-1bbf-4adc-959d-7e4b0c2d1dd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:31.605788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c35f37-1bbf-4adc-959d-7e4b0c2d1dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:31.607236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c35f37-1bbf-4adc-959d-7e4b0c2d1dd3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:31.617201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 537/1068] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:31.623656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed63c9-bcca-4333-b867-1df91d1ae847 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:31.723801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:31.731930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed63c9-bcca-4333-b867-1df91d1ae847 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:31.767735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f5be9-4da1-422a-8d58-a5c7e5710a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:31.768609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f5be9-4da1-422a-8d58-a5c7e5710a8c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:31.768889 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f5be9-4da1-422a-8d58-a5c7e5710a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:31.769163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f5be9-4da1-422a-8d58-a5c7e5710a8c None admin] PATH_INFO: `/v3/users/7a77e35ff254415f830fd194c1714d3e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:31.783216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f5be9-4da1-422a-8d58-a5c7e5710a8c None admin] RBAC: Authorizing `identity:get_user(user_id=7a77e35ff254415f830fd194c1714d3e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:31.784651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f5be9-4da1-422a-8d58-a5c7e5710a8c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:31.786506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 532/1069] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:18:31 2022] GET /identity/v3/users/7a77e35ff254415f830fd194c1714d3e => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:31.793082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c35f37-1bbf-4adc-959d-7e4b0c2d1dd3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:31.892455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:31.896176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c35f37-1bbf-4adc-959d-7e4b0c2d1dd3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:31.931721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979507d2-0334-49bb-9bef-31cdf957370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:31.932377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979507d2-0334-49bb-9bef-31cdf957370d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:31.932773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979507d2-0334-49bb-9bef-31cdf957370d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:31.933182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979507d2-0334-49bb-9bef-31cdf957370d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:32.002602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979507d2-0334-49bb-9bef-31cdf957370d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:32.004060 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979507d2-0334-49bb-9bef-31cdf957370d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:32.106029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:32.110320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 538/1070] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 06:18:32.116624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f5be9-4da1-422a-8d58-a5c7e5710a8c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:32.232755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f5be9-4da1-422a-8d58-a5c7e5710a8c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:32.273664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e0316-4a3d-419b-a10b-6a1570d1ee7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:32.274566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e0316-4a3d-419b-a10b-6a1570d1ee7b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:32.274850 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e0316-4a3d-419b-a10b-6a1570d1ee7b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:32.275131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e0316-4a3d-419b-a10b-6a1570d1ee7b None admin] PATH_INFO: `/v3/projects/46033645552d4c0a86dad89e2d78816b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:32.276362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e0316-4a3d-419b-a10b-6a1570d1ee7b None admin] RBAC: Authorizing `identity:get_project(project_id=46033645552d4c0a86dad89e2d78816b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:32.277794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e0316-4a3d-419b-a10b-6a1570d1ee7b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:32.284271 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 533/1071] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:18:32 2022] GET /identity/v3/projects/46033645552d4c0a86dad89e2d78816b => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:32.285672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979507d2-0334-49bb-9bef-31cdf957370d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:32.390116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979507d2-0334-49bb-9bef-31cdf957370d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:32.425879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267191-2597-4609-8eb5-37c1636ef534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:32.426568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267191-2597-4609-8eb5-37c1636ef534 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:32.426853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267191-2597-4609-8eb5-37c1636ef534 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:32.427130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267191-2597-4609-8eb5-37c1636ef534 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:32.427963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48267191-2597-4609-8eb5-37c1636ef534 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=46033645552d4c0a86dad89e2d78816b, user.id=7a77e35ff254415f830fd194c1714d3e) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:32.428275 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48267191-2597-4609-8eb5-37c1636ef534 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:32.429704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48267191-2597-4609-8eb5-37c1636ef534 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:32.437987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 539/1072] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:18:32 2022] GET /identity/v3/role_assignments?user.id=7a77e35ff254415f830fd194c1714d3e&scope.project.id=46033645552d4c0a86dad89e2d78816b&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:32.615072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cbb62-fd57-4cb2-81f4-e76ea9a11a2f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:32.615424 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cbb62-fd57-4cb2-81f4-e76ea9a11a2f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:32.615545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cbb62-fd57-4cb2-81f4-e76ea9a11a2f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:32.617310 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 534/1073] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:32.623167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2e8ab-488d-491b-9723-ad4b827bf074 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:32.623475 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2e8ab-488d-491b-9723-ad4b827bf074 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:32.623742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2e8ab-488d-491b-9723-ad4b827bf074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:32.641003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-19d2e8ab-488d-491b-9723-ad4b827bf074 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:32.642236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-19d2e8ab-488d-491b-9723-ad4b827bf074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:32.705192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:32.715954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 540/1074] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:32.736853 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cbb62-fd57-4cb2-81f4-e76ea9a11a2f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:32.769966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:32.843460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cbb62-fd57-4cb2-81f4-e76ea9a11a2f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:32.878296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc35c3-e747-41b6-9361-06945c2946f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:32.879039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc35c3-e747-41b6-9361-06945c2946f4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:32.879323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc35c3-e747-41b6-9361-06945c2946f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:32.879603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc35c3-e747-41b6-9361-06945c2946f4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:32.891047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc35c3-e747-41b6-9361-06945c2946f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5ebc35c3-e747-41b6-9361-06945c2946f4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:32.899618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:32.900567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:32.901473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 535/1075] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 06:18:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:32.906881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2e8ab-488d-491b-9723-ad4b827bf074 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:33.007215 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2e8ab-488d-491b-9723-ad4b827bf074 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:33.041035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce381722-9737-46e0-b2a4-65b5d8f23c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:33.041714 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce381722-9737-46e0-b2a4-65b5d8f23c39 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:33.042650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce381722-9737-46e0-b2a4-65b5d8f23c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:33.042934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce381722-9737-46e0-b2a4-65b5d8f23c39 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:33.043699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce381722-9737-46e0-b2a4-65b5d8f23c39 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:33.044015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce381722-9737-46e0-b2a4-65b5d8f23c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:33.045394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce381722-9737-46e0-b2a4-65b5d8f23c39 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:33.062263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 541/1076] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 06:18:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:33.063383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc35c3-e747-41b6-9361-06945c2946f4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:33.147413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:33.167859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc35c3-e747-41b6-9361-06945c2946f4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:33.202501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5459cf4-6cbb-4b76-85a6-3cf2760c7ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:33.203302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459cf4-6cbb-4b76-85a6-3cf2760c7ddd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:33.203589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459cf4-6cbb-4b76-85a6-3cf2760c7ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:33.203870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459cf4-6cbb-4b76-85a6-3cf2760c7ddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:33.214641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5459cf4-6cbb-4b76-85a6-3cf2760c7ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:33.216083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5459cf4-6cbb-4b76-85a6-3cf2760c7ddd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c5459cf4-6cbb-4b76-85a6-3cf2760c7ddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:33.233607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:33.234730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:33.235618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:33.235618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:33.235618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:33.235618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:33.238270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 536/1077] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:33.240495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce381722-9737-46e0-b2a4-65b5d8f23c39 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:33.299201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:33.346232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce381722-9737-46e0-b2a4-65b5d8f23c39 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:33.380457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae1d77-9da3-4fc5-865f-e3b37132289e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:33.381122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae1d77-9da3-4fc5-865f-e3b37132289e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:33.381406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae1d77-9da3-4fc5-865f-e3b37132289e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:33.381682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae1d77-9da3-4fc5-865f-e3b37132289e None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:33.382431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae1d77-9da3-4fc5-865f-e3b37132289e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:33.382741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae1d77-9da3-4fc5-865f-e3b37132289e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:33.384098 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae1d77-9da3-4fc5-865f-e3b37132289e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:33.400677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 542/1078] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:33.402637 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5459cf4-6cbb-4b76-85a6-3cf2760c7ddd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:33.504456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5459cf4-6cbb-4b76-85a6-3cf2760c7ddd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:33.538634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2992967-7fae-439d-96bc-279bd8564e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:33.539437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2992967-7fae-439d-96bc-279bd8564e9c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:33.539725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2992967-7fae-439d-96bc-279bd8564e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:33.540006 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2992967-7fae-439d-96bc-279bd8564e9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:33.543105 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:33.550803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2992967-7fae-439d-96bc-279bd8564e9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:33.552292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2992967-7fae-439d-96bc-279bd8564e9c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:33.553320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8026e-3d2e-41d0-b128-bd786b180e69 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:33.553611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8026e-3d2e-41d0-b128-bd786b180e69 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:33.553888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8026e-3d2e-41d0-b128-bd786b180e69 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:33.556831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 543/1079] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:33.561469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b97be-73c9-4154-8190-51bac62de3e9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:33.561751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b97be-73c9-4154-8190-51bac62de3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:33.562077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b97be-73c9-4154-8190-51bac62de3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d2992967-7fae-439d-96bc-279bd8564e9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:18:33.568157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:33.569334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 537/1080] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 06:18:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:33.575990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2992967-7fae-439d-96bc-279bd8564e9c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:33.587697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b16b97be-73c9-4154-8190-51bac62de3e9 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:33.589537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b16b97be-73c9-4154-8190-51bac62de3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:33.660623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 544/1081] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:33.683098 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b97be-73c9-4154-8190-51bac62de3e9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:33.690263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2992967-7fae-439d-96bc-279bd8564e9c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:33.728955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da542f-396d-48ba-8c38-a4efe2a8ccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:33.730083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da542f-396d-48ba-8c38-a4efe2a8ccad None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:33.730409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da542f-396d-48ba-8c38-a4efe2a8ccad None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:33.730660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da542f-396d-48ba-8c38-a4efe2a8ccad None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:33.731511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da542f-396d-48ba-8c38-a4efe2a8ccad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:33.731829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da542f-396d-48ba-8c38-a4efe2a8ccad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:33.732171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:33.733663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da542f-396d-48ba-8c38-a4efe2a8ccad None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:33.751641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 538/1082] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 06:18:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:33.757174 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da542f-396d-48ba-8c38-a4efe2a8ccad None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:33.794266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b97be-73c9-4154-8190-51bac62de3e9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:33.816616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:33.831674 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6365f-0375-49fa-b826-e1f0d98d96de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:33.832327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6365f-0375-49fa-b826-e1f0d98d96de None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:33.832609 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6365f-0375-49fa-b826-e1f0d98d96de None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:33.832884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6365f-0375-49fa-b826-e1f0d98d96de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:33.868805 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da542f-396d-48ba-8c38-a4efe2a8ccad None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:33.905262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f81cd7-af84-423c-b39d-348fd7e1880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:33.905915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f81cd7-af84-423c-b39d-348fd7e1880d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:33.906229 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f81cd7-af84-423c-b39d-348fd7e1880d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:33.906509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f81cd7-af84-423c-b39d-348fd7e1880d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:33.907481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6365f-0375-49fa-b826-e1f0d98d96de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:33.909089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6365f-0375-49fa-b826-e1f0d98d96de None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:33.955501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:33.980971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f81cd7-af84-423c-b39d-348fd7e1880d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:33.982539 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f81cd7-af84-423c-b39d-348fd7e1880d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:34.032072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 545/1083] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 06:18:34.033730 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6365f-0375-49fa-b826-e1f0d98d96de None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:34.035424 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:34.090168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 539/1084] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:18:34.095673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f81cd7-af84-423c-b39d-348fd7e1880d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:34.152420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6365f-0375-49fa-b826-e1f0d98d96de None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:34.189083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3888-6d9d-4744-b910-60079b6496a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:34.190002 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3888-6d9d-4744-b910-60079b6496a8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:34.190292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3888-6d9d-4744-b910-60079b6496a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:34.190572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3888-6d9d-4744-b910-60079b6496a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:34.201737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b3888-6d9d-4744-b910-60079b6496a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:34.202482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f81cd7-af84-423c-b39d-348fd7e1880d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:34.203218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b3888-6d9d-4744-b910-60079b6496a8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:34.211454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9e5b3888-6d9d-4744-b910-60079b6496a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:34.223294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:34.224220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:34.225127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:34.225127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:34.225127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:34.225127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:34.225127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 546/1085] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:34.230916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3888-6d9d-4744-b910-60079b6496a8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:34.240487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c1708-1e67-46fc-b8aa-90170f749343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:34.241293 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c1708-1e67-46fc-b8aa-90170f749343 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:34.241578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c1708-1e67-46fc-b8aa-90170f749343 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:34.241857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c1708-1e67-46fc-b8aa-90170f749343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:34.253571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c1708-1e67-46fc-b8aa-90170f749343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:34.255030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c1708-1e67-46fc-b8aa-90170f749343 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fb9c1708-1e67-46fc-b8aa-90170f749343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:18:34.273368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:34.274322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:34.280030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:18:34.280030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:18:34.280030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:18:34.280030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:34.280030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 540/1086] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:34.280884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c1708-1e67-46fc-b8aa-90170f749343 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:34.284387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:34.347241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3888-6d9d-4744-b910-60079b6496a8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:34.382385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b84cb-4d12-4da5-a4ee-ee0df9ec9a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:34.383050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b84cb-4d12-4da5-a4ee-ee0df9ec9a5a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:34.383334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b84cb-4d12-4da5-a4ee-ee0df9ec9a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:34.383609 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b84cb-4d12-4da5-a4ee-ee0df9ec9a5a None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:34.384391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b84cb-4d12-4da5-a4ee-ee0df9ec9a5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:34.384697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b84cb-4d12-4da5-a4ee-ee0df9ec9a5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:34.387118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b84cb-4d12-4da5-a4ee-ee0df9ec9a5a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:34.395530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c1708-1e67-46fc-b8aa-90170f749343 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:34.399530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 547/1087] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:34.434177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554a382-dc70-44e4-a2ec-4851f0401c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:34.434882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554a382-dc70-44e4-a2ec-4851f0401c16 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:34.435167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554a382-dc70-44e4-a2ec-4851f0401c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:34.435450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554a382-dc70-44e4-a2ec-4851f0401c16 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:34.436204 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554a382-dc70-44e4-a2ec-4851f0401c16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:34.436518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554a382-dc70-44e4-a2ec-4851f0401c16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:34.437910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554a382-dc70-44e4-a2ec-4851f0401c16 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:34.456737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 541/1088] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:34.458366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b84cb-4d12-4da5-a4ee-ee0df9ec9a5a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:34.482069 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc2e0b-d21e-4963-880b-0624fbf24da2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:34.482406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc2e0b-d21e-4963-880b-0624fbf24da2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:34.482639 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc2e0b-d21e-4963-880b-0624fbf24da2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:34.484289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 542/1089] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:18:34.489702 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7159b-5807-4ca4-af16-9a897e4dda27 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:34.490028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7159b-5807-4ca4-af16-9a897e4dda27 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:34.490323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7159b-5807-4ca4-af16-9a897e4dda27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:34.508318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-12a7159b-5807-4ca4-af16-9a897e4dda27 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:34.509536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-12a7159b-5807-4ca4-af16-9a897e4dda27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:34.531754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:34.580174 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b84cb-4d12-4da5-a4ee-ee0df9ec9a5a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:34.587655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:34.599409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 543/1090] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:18:34.617121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6052027-706b-4b5d-bae0-bb2ae6d7cfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:34.617929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6052027-706b-4b5d-bae0-bb2ae6d7cfdc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:34.618242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6052027-706b-4b5d-bae0-bb2ae6d7cfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:34.618518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6052027-706b-4b5d-bae0-bb2ae6d7cfdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:34.623153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7159b-5807-4ca4-af16-9a897e4dda27 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:34.626449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6052027-706b-4b5d-bae0-bb2ae6d7cfdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:34.627914 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6052027-706b-4b5d-bae0-bb2ae6d7cfdc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e6052027-706b-4b5d-bae0-bb2ae6d7cfdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:18:34.642987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:34.643925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:34.644847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:18:34.644847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:18:34.644847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:18:34.644847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:18:34.644847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:18:34.644847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:18:34.644847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:18:34.644847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:34.644847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 548/1091] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:34.651775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6052027-706b-4b5d-bae0-bb2ae6d7cfdc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:34.733143 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7159b-5807-4ca4-af16-9a897e4dda27 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:34.755777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6052027-706b-4b5d-bae0-bb2ae6d7cfdc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:34.766963 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d741-a823-4860-8ad1-ed9e7307a6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:34.767750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d741-a823-4860-8ad1-ed9e7307a6ce None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:34.768035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d741-a823-4860-8ad1-ed9e7307a6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:34.768310 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d741-a823-4860-8ad1-ed9e7307a6ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:34.773834 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:34.778983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8d741-a823-4860-8ad1-ed9e7307a6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c1c8d741-a823-4860-8ad1-ed9e7307a6ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:34.786666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:34.787720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:18:34.788708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 544/1092] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:18:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:18:34.791599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474e27f-58bd-4046-9c1f-e187f421b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:34.792830 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474e27f-58bd-4046-9c1f-e187f421b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:34.793109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474e27f-58bd-4046-9c1f-e187f421b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:34.793380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474e27f-58bd-4046-9c1f-e187f421b1a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:34.794201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474e27f-58bd-4046-9c1f-e187f421b1a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:34.794510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474e27f-58bd-4046-9c1f-e187f421b1a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:34.794697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d741-a823-4860-8ad1-ed9e7307a6ce None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:34.796026 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474e27f-58bd-4046-9c1f-e187f421b1a7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:34.809345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 549/1093] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:18:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:34.815204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474e27f-58bd-4046-9c1f-e187f421b1a7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:34.851450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:34.899096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d741-a823-4860-8ad1-ed9e7307a6ce None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:34.916591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474e27f-58bd-4046-9c1f-e187f421b1a7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:34.933103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e51bb-4172-4eae-a2c1-45b22528b2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:34.933772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e51bb-4172-4eae-a2c1-45b22528b2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:34.934099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e51bb-4172-4eae-a2c1-45b22528b2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:34.934389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e51bb-4172-4eae-a2c1-45b22528b2d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:34.935142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e51bb-4172-4eae-a2c1-45b22528b2d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:34.935450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e51bb-4172-4eae-a2c1-45b22528b2d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:34.936828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e51bb-4172-4eae-a2c1-45b22528b2d8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:34.944984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 545/1094] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:18:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:34.950753 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e51bb-4172-4eae-a2c1-45b22528b2d8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:34.950897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd42207-e774-4d1c-b215-ac14a7f417a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:34.951515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd42207-e774-4d1c-b215-ac14a7f417a8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:34.951796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd42207-e774-4d1c-b215-ac14a7f417a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:34.952072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd42207-e774-4d1c-b215-ac14a7f417a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:34.952903 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd42207-e774-4d1c-b215-ac14a7f417a8 None admin] RBAC: Adding query filter params (role.id=d726d771b3d34d378b43824cbfec19c9, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=058f099e6f7545a1b2734dbe5afd2027) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:34.953216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd42207-e774-4d1c-b215-ac14a7f417a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:34.954671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd42207-e774-4d1c-b215-ac14a7f417a8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:34.960669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 550/1095] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:18:34 2022] GET /identity/v3/role_assignments?user.id=058f099e6f7545a1b2734dbe5afd2027&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=d726d771b3d34d378b43824cbfec19c9 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:35.063326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e51bb-4172-4eae-a2c1-45b22528b2d8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:35.081022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:35.100144 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59869355-72fa-4544-9b29-9d56dea1e1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:35.101028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59869355-72fa-4544-9b29-9d56dea1e1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:35.101307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59869355-72fa-4544-9b29-9d56dea1e1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:35.101580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59869355-72fa-4544-9b29-9d56dea1e1b2 None admin] PATH_INFO: `/v3/users/7a77e35ff254415f830fd194c1714d3e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:35.102807 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59869355-72fa-4544-9b29-9d56dea1e1b2 None admin] RBAC: Authorizing `identity:get_user(user_id=7a77e35ff254415f830fd194c1714d3e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:35.104267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59869355-72fa-4544-9b29-9d56dea1e1b2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:35.107130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 546/1096] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:18:34 2022] GET /identity/v3/users/7a77e35ff254415f830fd194c1714d3e => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:35.113501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd42207-e774-4d1c-b215-ac14a7f417a8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:35.160025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:35.218519 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd42207-e774-4d1c-b215-ac14a7f417a8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:35.251949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0dfa2-110e-48ed-b97a-3b6e490c11f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:35.252817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0dfa2-110e-48ed-b97a-3b6e490c11f2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:35.253099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0dfa2-110e-48ed-b97a-3b6e490c11f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:35.253375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0dfa2-110e-48ed-b97a-3b6e490c11f2 None admin] PATH_INFO: `/v3/projects/46033645552d4c0a86dad89e2d78816b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:35.254580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0dfa2-110e-48ed-b97a-3b6e490c11f2 None admin] RBAC: Authorizing `identity:get_project(project_id=46033645552d4c0a86dad89e2d78816b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:35.256017 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0dfa2-110e-48ed-b97a-3b6e490c11f2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:35.258056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 551/1097] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:18:35 2022] GET /identity/v3/projects/46033645552d4c0a86dad89e2d78816b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:35.264548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59869355-72fa-4544-9b29-9d56dea1e1b2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:35.367787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59869355-72fa-4544-9b29-9d56dea1e1b2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:35.402361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb0faf-aec4-4026-b05b-3345942dad63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:35.403263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb0faf-aec4-4026-b05b-3345942dad63 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:35.403546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb0faf-aec4-4026-b05b-3345942dad63 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:35.403823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb0faf-aec4-4026-b05b-3345942dad63 None admin] PATH_INFO: `/v3/projects/46033645552d4c0a86dad89e2d78816b/users/7a77e35ff254415f830fd194c1714d3e/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:35.405770 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb0faf-aec4-4026-b05b-3345942dad63 None admin] RBAC: Authorizing `identity:create_grant(project_id=46033645552d4c0a86dad89e2d78816b, user_id=7a77e35ff254415f830fd194c1714d3e, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:35.407323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb0faf-aec4-4026-b05b-3345942dad63 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:35.421555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 547/1098] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:18:35 2022] PUT /identity/v3/projects/46033645552d4c0a86dad89e2d78816b/users/7a77e35ff254415f830fd194c1714d3e/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:18:37.464402 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c639d2-bc02-4d35-8010-6a0bfc3a99f6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:37.464929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c639d2-bc02-4d35-8010-6a0bfc3a99f6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:37.464929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c639d2-bc02-4d35-8010-6a0bfc3a99f6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:37.466272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 552/1099] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:37.471378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c2e24-8409-4522-856b-d1d2e0a46402 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:37.471757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c2e24-8409-4522-856b-d1d2e0a46402 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:37.471946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c2e24-8409-4522-856b-d1d2e0a46402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:37.488996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-982c2e24-8409-4522-856b-d1d2e0a46402 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:37.490206 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-982c2e24-8409-4522-856b-d1d2e0a46402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:37.529113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:37.555640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 548/1100] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:37.580078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c639d2-bc02-4d35-8010-6a0bfc3a99f6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:37.680641 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c639d2-bc02-4d35-8010-6a0bfc3a99f6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:37.691671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:37.716041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f7826-bed8-4947-a281-1e23cefdc477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:37.716768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f7826-bed8-4947-a281-1e23cefdc477 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:37.717057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f7826-bed8-4947-a281-1e23cefdc477 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:37.717338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f7826-bed8-4947-a281-1e23cefdc477 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:37.727613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f7826-bed8-4947-a281-1e23cefdc477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-282f7826-bed8-4947-a281-1e23cefdc477 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:37.735314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:37.736378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:37.737352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 553/1101] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:18:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:37.742579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c2e24-8409-4522-856b-d1d2e0a46402 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:37.840401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c2e24-8409-4522-856b-d1d2e0a46402 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:37.873015 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d027bc-1dc3-4f13-b4a5-51c585a679df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:37.873664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d027bc-1dc3-4f13-b4a5-51c585a679df None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:37.873979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d027bc-1dc3-4f13-b4a5-51c585a679df None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:37.874258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d027bc-1dc3-4f13-b4a5-51c585a679df None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:37.875009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d027bc-1dc3-4f13-b4a5-51c585a679df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:37.875319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d027bc-1dc3-4f13-b4a5-51c585a679df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:37.876663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d027bc-1dc3-4f13-b4a5-51c585a679df None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:37.884434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 549/1102] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:18:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:37.890917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f7826-bed8-4947-a281-1e23cefdc477 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:37.988346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f7826-bed8-4947-a281-1e23cefdc477 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:38.022487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12643080-f613-4878-b04e-b7665c44a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:38.023341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12643080-f613-4878-b04e-b7665c44a6aa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:38.023621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12643080-f613-4878-b04e-b7665c44a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:38.023892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12643080-f613-4878-b04e-b7665c44a6aa None admin] PATH_INFO: `/v3/users/7a77e35ff254415f830fd194c1714d3e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:38.025058 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12643080-f613-4878-b04e-b7665c44a6aa None admin] RBAC: Authorizing `identity:get_user(user_id=7a77e35ff254415f830fd194c1714d3e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:38.027443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12643080-f613-4878-b04e-b7665c44a6aa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:38.029029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 554/1103] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:18:37 2022] GET /identity/v3/users/7a77e35ff254415f830fd194c1714d3e => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:38.034733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d027bc-1dc3-4f13-b4a5-51c585a679df None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:38.070281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:38.134047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d027bc-1dc3-4f13-b4a5-51c585a679df None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:38.166452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3786f8-dfdb-477b-8e45-557c6669ecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:38.167726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3786f8-dfdb-477b-8e45-557c6669ecd5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:38.167987 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3786f8-dfdb-477b-8e45-557c6669ecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:38.168268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3786f8-dfdb-477b-8e45-557c6669ecd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:38.235167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3786f8-dfdb-477b-8e45-557c6669ecd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:38.236603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3786f8-dfdb-477b-8e45-557c6669ecd5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:38.268677 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:38.331671 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 550/1104] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:18:38.337523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12643080-f613-4878-b04e-b7665c44a6aa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:38.376260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:38.435743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12643080-f613-4878-b04e-b7665c44a6aa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:38.468828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2b77c-74c5-4f7d-9609-66743e0e8eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:38.469678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b77c-74c5-4f7d-9609-66743e0e8eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:38.469977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b77c-74c5-4f7d-9609-66743e0e8eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:38.470260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b77c-74c5-4f7d-9609-66743e0e8eb0 None admin] PATH_INFO: `/v3/projects/46033645552d4c0a86dad89e2d78816b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:38.471426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2b77c-74c5-4f7d-9609-66743e0e8eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=46033645552d4c0a86dad89e2d78816b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:38.472850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2b77c-74c5-4f7d-9609-66743e0e8eb0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:38.474548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 555/1105] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:18:38 2022] GET /identity/v3/projects/46033645552d4c0a86dad89e2d78816b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:38.481276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3786f8-dfdb-477b-8e45-557c6669ecd5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:38.576796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3786f8-dfdb-477b-8e45-557c6669ecd5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:38.610530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b552e63-153d-4f9e-870b-67615627e334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:38.611182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b552e63-153d-4f9e-870b-67615627e334 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:38.611470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b552e63-153d-4f9e-870b-67615627e334 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:38.611751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b552e63-153d-4f9e-870b-67615627e334 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:38.612583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b552e63-153d-4f9e-870b-67615627e334 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=46033645552d4c0a86dad89e2d78816b, user.id=7a77e35ff254415f830fd194c1714d3e) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:38.612898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b552e63-153d-4f9e-870b-67615627e334 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:38.614357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b552e63-153d-4f9e-870b-67615627e334 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:38.617112 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:38.620484 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 551/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:18:38 2022] GET /identity/v3/role_assignments?user.id=7a77e35ff254415f830fd194c1714d3e&scope.project.id=46033645552d4c0a86dad89e2d78816b&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:18:40.665370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334c971-caa3-4761-a14c-36a706f74122 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:40.665979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334c971-caa3-4761-a14c-36a706f74122 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:40.665979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334c971-caa3-4761-a14c-36a706f74122 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:40.667925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 556/1107] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:40.673077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15136-101e-48ab-9028-b2450d5ed733 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:40.673520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15136-101e-48ab-9028-b2450d5ed733 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:40.673649 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15136-101e-48ab-9028-b2450d5ed733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:40.690679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-28f15136-101e-48ab-9028-b2450d5ed733 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:40.691837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-28f15136-101e-48ab-9028-b2450d5ed733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:40.756838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 552/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:40.781376 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334c971-caa3-4761-a14c-36a706f74122 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:40.880013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334c971-caa3-4761-a14c-36a706f74122 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:40.887122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:40.914728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0dbe3-8b7b-4a66-bc33-09431ff36bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:40.915565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0dbe3-8b7b-4a66-bc33-09431ff36bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:40.915802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0dbe3-8b7b-4a66-bc33-09431ff36bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:40.916076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0dbe3-8b7b-4a66-bc33-09431ff36bb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:40.918047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0dbe3-8b7b-4a66-bc33-09431ff36bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:40.919477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0dbe3-8b7b-4a66-bc33-09431ff36bb3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:40.921074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 557/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:40.926672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f15136-101e-48ab-9028-b2450d5ed733 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:41.024303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f15136-101e-48ab-9028-b2450d5ed733 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:41.053489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:41.059788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdffe93-6938-44c3-9225-eba3a97b360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:41.060468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdffe93-6938-44c3-9225-eba3a97b360c None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:41.060757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdffe93-6938-44c3-9225-eba3a97b360c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:41.061039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdffe93-6938-44c3-9225-eba3a97b360c None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:41.061764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdffe93-6938-44c3-9225-eba3a97b360c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:41.063239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdffe93-6938-44c3-9225-eba3a97b360c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:41.082793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 553/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:18:40 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:18:43.138476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60d8fa-fdec-4656-93ad-64c4f1199769 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:43.139020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60d8fa-fdec-4656-93ad-64c4f1199769 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:43.139020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60d8fa-fdec-4656-93ad-64c4f1199769 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:43.140380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 558/1111] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:43.145590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cdfe8-e90f-4157-8c8f-db352f349b89 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:43.146044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cdfe8-e90f-4157-8c8f-db352f349b89 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:43.146178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cdfe8-e90f-4157-8c8f-db352f349b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:43.163625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b74cdfe8-e90f-4157-8c8f-db352f349b89 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:43.164774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b74cdfe8-e90f-4157-8c8f-db352f349b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:43.230567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 554/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:43.252918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60d8fa-fdec-4656-93ad-64c4f1199769 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:43.351563 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60d8fa-fdec-4656-93ad-64c4f1199769 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:43.388284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909d614-fd5a-4550-8ac6-aabf2b8d4bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:43.389119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909d614-fd5a-4550-8ac6-aabf2b8d4bcb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:43.389400 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909d614-fd5a-4550-8ac6-aabf2b8d4bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:43.389676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909d614-fd5a-4550-8ac6-aabf2b8d4bcb None admin] PATH_INFO: `/v3/roles/d534ab3d0a904f0babeb90d2679d1b86` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:43.391180 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909d614-fd5a-4550-8ac6-aabf2b8d4bcb None admin] RBAC: Authorizing `identity:get_role(role_id=d534ab3d0a904f0babeb90d2679d1b86)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:43.392554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909d614-fd5a-4550-8ac6-aabf2b8d4bcb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:43.393907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 559/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:18:43 2022] GET /identity/v3/roles/d534ab3d0a904f0babeb90d2679d1b86 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:43.399931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cdfe8-e90f-4157-8c8f-db352f349b89 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:43.498582 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cdfe8-e90f-4157-8c8f-db352f349b89 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:43.515148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:43.534159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce0504-1660-4fb4-81c5-25086e19515f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:43.535020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce0504-1660-4fb4-81c5-25086e19515f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:43.535305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce0504-1660-4fb4-81c5-25086e19515f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:43.535581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce0504-1660-4fb4-81c5-25086e19515f None admin] PATH_INFO: `/v3/users/c3807f5d8c264969a3ae395272764f3a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:43.548174 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce0504-1660-4fb4-81c5-25086e19515f None admin] RBAC: Authorizing `identity:get_user(user_id=c3807f5d8c264969a3ae395272764f3a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:43.549601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce0504-1660-4fb4-81c5-25086e19515f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:43.551271 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 555/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:18:43 2022] GET /identity/v3/users/c3807f5d8c264969a3ae395272764f3a => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:43.557002 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909d614-fd5a-4550-8ac6-aabf2b8d4bcb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:43.600949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:43.655650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909d614-fd5a-4550-8ac6-aabf2b8d4bcb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:43.687760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ed13c-c83a-4838-951e-d4af97bf57f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:43.688398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ed13c-c83a-4838-951e-d4af97bf57f9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:43.688682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ed13c-c83a-4838-951e-d4af97bf57f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:43.688956 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ed13c-c83a-4838-951e-d4af97bf57f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:43.753633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ed13c-c83a-4838-951e-d4af97bf57f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:43.755167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ed13c-c83a-4838-951e-d4af97bf57f9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:43.795992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:43.852720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 560/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:18:43.858757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce0504-1660-4fb4-81c5-25086e19515f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:43.955893 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce0504-1660-4fb4-81c5-25086e19515f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:43.987733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c64a7-9dfb-411a-a7de-07da202e23df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:43.988575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c64a7-9dfb-411a-a7de-07da202e23df None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:43.988859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c64a7-9dfb-411a-a7de-07da202e23df None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:43.989135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c64a7-9dfb-411a-a7de-07da202e23df None admin] PATH_INFO: `/v3/projects/46033645552d4c0a86dad89e2d78816b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:43.990340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c64a7-9dfb-411a-a7de-07da202e23df None admin] RBAC: Authorizing `identity:get_project(project_id=46033645552d4c0a86dad89e2d78816b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:43.991744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c64a7-9dfb-411a-a7de-07da202e23df None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:43.993402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 556/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:18:43 2022] GET /identity/v3/projects/46033645552d4c0a86dad89e2d78816b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:43.999816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ed13c-c83a-4838-951e-d4af97bf57f9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:44.097145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ed13c-c83a-4838-951e-d4af97bf57f9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:44.129979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca628fe9-2584-4688-aec6-6e0aeef97a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:44.130631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca628fe9-2584-4688-aec6-6e0aeef97a50 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:44.130916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca628fe9-2584-4688-aec6-6e0aeef97a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:44.131192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca628fe9-2584-4688-aec6-6e0aeef97a50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:44.132019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca628fe9-2584-4688-aec6-6e0aeef97a50 None admin] RBAC: Adding query filter params (role.id=d534ab3d0a904f0babeb90d2679d1b86, scope.project.id=46033645552d4c0a86dad89e2d78816b, user.id=c3807f5d8c264969a3ae395272764f3a) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:44.132329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca628fe9-2584-4688-aec6-6e0aeef97a50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:44.133754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca628fe9-2584-4688-aec6-6e0aeef97a50 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:44.139808 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 561/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:18:43 2022] GET /identity/v3/role_assignments?user.id=c3807f5d8c264969a3ae395272764f3a&scope.project.id=46033645552d4c0a86dad89e2d78816b&role.id=d534ab3d0a904f0babeb90d2679d1b86 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:46.163959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc1fa-8f95-49d4-a15e-708417beb4c2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:46.164456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc1fa-8f95-49d4-a15e-708417beb4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:46.164572 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc1fa-8f95-49d4-a15e-708417beb4c2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:46.165887 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 557/1118] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:46.171924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3120cec-c72c-47c1-a1fc-6043d8f29b18 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:46.172351 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3120cec-c72c-47c1-a1fc-6043d8f29b18 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:46.172480 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3120cec-c72c-47c1-a1fc-6043d8f29b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:46.183894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:46.190739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f3120cec-c72c-47c1-a1fc-6043d8f29b18 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:46.191947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f3120cec-c72c-47c1-a1fc-6043d8f29b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:46.258230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 562/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:46.281583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabc1fa-8f95-49d4-a15e-708417beb4c2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:46.325525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:46.380741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabc1fa-8f95-49d4-a15e-708417beb4c2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:46.413954 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4404a-818b-4121-a90b-3d6ede65ceae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:46.414811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4404a-818b-4121-a90b-3d6ede65ceae None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:46.415094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4404a-818b-4121-a90b-3d6ede65ceae None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:46.415369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4404a-818b-4121-a90b-3d6ede65ceae None admin] PATH_INFO: `/v3/roles/d534ab3d0a904f0babeb90d2679d1b86` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:46.417372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4404a-818b-4121-a90b-3d6ede65ceae None admin] RBAC: Authorizing `identity:get_role(role_id=d534ab3d0a904f0babeb90d2679d1b86)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:46.418849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4404a-818b-4121-a90b-3d6ede65ceae None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:46.420213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 558/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:18:46 2022] GET /identity/v3/roles/d534ab3d0a904f0babeb90d2679d1b86 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:46.426102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3120cec-c72c-47c1-a1fc-6043d8f29b18 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:46.522451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3120cec-c72c-47c1-a1fc-6043d8f29b18 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:46.555743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b612a3f-fe1e-4639-9342-b076f5b49ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:46.556605 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b612a3f-fe1e-4639-9342-b076f5b49ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:46.556886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b612a3f-fe1e-4639-9342-b076f5b49ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:46.557160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b612a3f-fe1e-4639-9342-b076f5b49ed7 None admin] PATH_INFO: `/v3/users/c3807f5d8c264969a3ae395272764f3a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:46.558461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b612a3f-fe1e-4639-9342-b076f5b49ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=c3807f5d8c264969a3ae395272764f3a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:46.559902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b612a3f-fe1e-4639-9342-b076f5b49ed7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:46.561509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 563/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:18:46 2022] GET /identity/v3/users/c3807f5d8c264969a3ae395272764f3a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:46.567363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4404a-818b-4121-a90b-3d6ede65ceae None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:46.664045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4404a-818b-4121-a90b-3d6ede65ceae None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:46.674250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:46.696869 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37178908-c445-4377-b2b6-a855dc466b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:46.697714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37178908-c445-4377-b2b6-a855dc466b23 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:46.698554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37178908-c445-4377-b2b6-a855dc466b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:46.698903 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37178908-c445-4377-b2b6-a855dc466b23 None admin] PATH_INFO: `/v3/projects/46033645552d4c0a86dad89e2d78816b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:46.700068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37178908-c445-4377-b2b6-a855dc466b23 None admin] RBAC: Authorizing `identity:get_project(project_id=46033645552d4c0a86dad89e2d78816b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:46.701489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37178908-c445-4377-b2b6-a855dc466b23 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:46.703192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 559/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:18:46 2022] GET /identity/v3/projects/46033645552d4c0a86dad89e2d78816b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:46.709107 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b612a3f-fe1e-4639-9342-b076f5b49ed7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:46.712826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:46.807171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b612a3f-fe1e-4639-9342-b076f5b49ed7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:46.841125 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147e2c9-056b-4241-b1d4-174de8fe5d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:46.842031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147e2c9-056b-4241-b1d4-174de8fe5d65 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:46.842315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147e2c9-056b-4241-b1d4-174de8fe5d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:46.842590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147e2c9-056b-4241-b1d4-174de8fe5d65 None admin] PATH_INFO: `/v3/projects/46033645552d4c0a86dad89e2d78816b/users/c3807f5d8c264969a3ae395272764f3a/roles/d534ab3d0a904f0babeb90d2679d1b86` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:46.844766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147e2c9-056b-4241-b1d4-174de8fe5d65 None admin] RBAC: Authorizing `identity:create_grant(project_id=46033645552d4c0a86dad89e2d78816b, user_id=c3807f5d8c264969a3ae395272764f3a, role_id=d534ab3d0a904f0babeb90d2679d1b86)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:46.846226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147e2c9-056b-4241-b1d4-174de8fe5d65 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:46.858831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 564/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:18:46 2022] PUT /identity/v3/projects/46033645552d4c0a86dad89e2d78816b/users/c3807f5d8c264969a3ae395272764f3a/roles/d534ab3d0a904f0babeb90d2679d1b86 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:18:48.849648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2b655-b0f1-4981-ab74-0637f27ab179 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:48.850232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2b655-b0f1-4981-ab74-0637f27ab179 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:48.850232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2b655-b0f1-4981-ab74-0637f27ab179 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:48.851592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 560/1124] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:48.857497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839716-0202-4cf1-81e7-9798463b848f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:48.857890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839716-0202-4cf1-81e7-9798463b848f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:48.858101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839716-0202-4cf1-81e7-9798463b848f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:48.875075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a2839716-0202-4cf1-81e7-9798463b848f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:48.876266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a2839716-0202-4cf1-81e7-9798463b848f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:48.934464 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:48.943378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 565/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:48.966659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2b655-b0f1-4981-ab74-0637f27ab179 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:49.065745 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2b655-b0f1-4981-ab74-0637f27ab179 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:49.098796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb43e2-26bc-4144-84d4-28897f8a4a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:49.099704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb43e2-26bc-4144-84d4-28897f8a4a15 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:49.099933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb43e2-26bc-4144-84d4-28897f8a4a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:49.100209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb43e2-26bc-4144-84d4-28897f8a4a15 None admin] PATH_INFO: `/v3/roles/d534ab3d0a904f0babeb90d2679d1b86` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:49.101671 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb43e2-26bc-4144-84d4-28897f8a4a15 None admin] RBAC: Authorizing `identity:get_role(role_id=d534ab3d0a904f0babeb90d2679d1b86)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:49.103066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb43e2-26bc-4144-84d4-28897f8a4a15 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:49.105128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 561/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:18:48 2022] GET /identity/v3/roles/d534ab3d0a904f0babeb90d2679d1b86 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:49.110891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2839716-0202-4cf1-81e7-9798463b848f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:49.206341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2839716-0202-4cf1-81e7-9798463b848f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:49.239573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10d800-fd69-47af-b1cb-298fefb87479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:49.240428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10d800-fd69-47af-b1cb-298fefb87479 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:49.240706 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10d800-fd69-47af-b1cb-298fefb87479 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:49.240978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10d800-fd69-47af-b1cb-298fefb87479 None admin] PATH_INFO: `/v3/users/c3807f5d8c264969a3ae395272764f3a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:49.242179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10d800-fd69-47af-b1cb-298fefb87479 None admin] RBAC: Authorizing `identity:get_user(user_id=c3807f5d8c264969a3ae395272764f3a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:49.243613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10d800-fd69-47af-b1cb-298fefb87479 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:49.245287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 566/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:18:49 2022] GET /identity/v3/users/c3807f5d8c264969a3ae395272764f3a => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:49.250766 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb43e2-26bc-4144-84d4-28897f8a4a15 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:49.297330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:49.350658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb43e2-26bc-4144-84d4-28897f8a4a15 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:49.383344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0a6e5-e9f8-4083-a2bd-d297fd119f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:49.383985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0a6e5-e9f8-4083-a2bd-d297fd119f8a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:49.384271 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0a6e5-e9f8-4083-a2bd-d297fd119f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:49.384551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0a6e5-e9f8-4083-a2bd-d297fd119f8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:49.450928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0a6e5-e9f8-4083-a2bd-d297fd119f8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:49.452368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0a6e5-e9f8-4083-a2bd-d297fd119f8a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:49.495872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:49.548796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 562/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:18:49.554657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10d800-fd69-47af-b1cb-298fefb87479 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:49.616525 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:49.655694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10d800-fd69-47af-b1cb-298fefb87479 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:49.688548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3b37c-02d8-4f22-a696-2844fa980449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:49.689491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3b37c-02d8-4f22-a696-2844fa980449 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:49.689784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3b37c-02d8-4f22-a696-2844fa980449 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:49.690092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3b37c-02d8-4f22-a696-2844fa980449 None admin] PATH_INFO: `/v3/projects/46033645552d4c0a86dad89e2d78816b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:49.691280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3b37c-02d8-4f22-a696-2844fa980449 None admin] RBAC: Authorizing `identity:get_project(project_id=46033645552d4c0a86dad89e2d78816b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:49.692716 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3b37c-02d8-4f22-a696-2844fa980449 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:49.694427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 567/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:18:49 2022] GET /identity/v3/projects/46033645552d4c0a86dad89e2d78816b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:49.701210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0a6e5-e9f8-4083-a2bd-d297fd119f8a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:49.798161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0a6e5-e9f8-4083-a2bd-d297fd119f8a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:49.831533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edd70e-3c1f-4c40-b204-2554a77b6a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:49.832170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edd70e-3c1f-4c40-b204-2554a77b6a8e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:49.832455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edd70e-3c1f-4c40-b204-2554a77b6a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:49.832732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edd70e-3c1f-4c40-b204-2554a77b6a8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:49.833558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93edd70e-3c1f-4c40-b204-2554a77b6a8e None admin] RBAC: Adding query filter params (role.id=d534ab3d0a904f0babeb90d2679d1b86, scope.project.id=46033645552d4c0a86dad89e2d78816b, user.id=c3807f5d8c264969a3ae395272764f3a) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:49.833870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93edd70e-3c1f-4c40-b204-2554a77b6a8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:49.835377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93edd70e-3c1f-4c40-b204-2554a77b6a8e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:49.841550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 563/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:18:49 2022] GET /identity/v3/role_assignments?user.id=c3807f5d8c264969a3ae395272764f3a&scope.project.id=46033645552d4c0a86dad89e2d78816b&role.id=d534ab3d0a904f0babeb90d2679d1b86 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:51.884837 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2fa77-bbd0-4ce4-9f87-720bced63834 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:51.885408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2fa77-bbd0-4ce4-9f87-720bced63834 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:51.885408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2fa77-bbd0-4ce4-9f87-720bced63834 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:51.887586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 568/1131] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:51.892629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea067a9-3e26-4491-85c4-a259feb77fdc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:51.892998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea067a9-3e26-4491-85c4-a259feb77fdc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:51.893198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea067a9-3e26-4491-85c4-a259feb77fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:51.909771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2ea067a9-3e26-4491-85c4-a259feb77fdc None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:51.910973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2ea067a9-3e26-4491-85c4-a259feb77fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:51.913654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:51.976058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 564/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:52.000427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2fa77-bbd0-4ce4-9f87-720bced63834 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:52.101791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2fa77-bbd0-4ce4-9f87-720bced63834 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:52.130840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:52.136911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf672905-6d5f-445c-b0b6-fec19cdd7e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:52.137637 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf672905-6d5f-445c-b0b6-fec19cdd7e9c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:52.137921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf672905-6d5f-445c-b0b6-fec19cdd7e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:52.138233 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf672905-6d5f-445c-b0b6-fec19cdd7e9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:52.139402 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf672905-6d5f-445c-b0b6-fec19cdd7e9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:52.140823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf672905-6d5f-445c-b0b6-fec19cdd7e9c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:52.142459 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 569/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:52.148561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea067a9-3e26-4491-85c4-a259feb77fdc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:52.243769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea067a9-3e26-4491-85c4-a259feb77fdc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:52.288940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd7225-a301-417c-8845-72c1c847ef27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:52.289595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd7225-a301-417c-8845-72c1c847ef27 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:52.289881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd7225-a301-417c-8845-72c1c847ef27 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:52.290188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd7225-a301-417c-8845-72c1c847ef27 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:52.290898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd7225-a301-417c-8845-72c1c847ef27 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:52.293314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd7225-a301-417c-8845-72c1c847ef27 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:52.302861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:52.318597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 565/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:18:52 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:18:54.382791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d44be9-dd4c-4655-9d44-071effc849ec None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:54.383289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d44be9-dd4c-4655-9d44-071effc849ec None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:54.383289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d44be9-dd4c-4655-9d44-071effc849ec None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:54.384594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 570/1135] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:54.390867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5312b8c-e53c-486d-9d0e-a7df49345926 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:54.391274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5312b8c-e53c-486d-9d0e-a7df49345926 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:54.391439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5312b8c-e53c-486d-9d0e-a7df49345926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:54.408050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c5312b8c-e53c-486d-9d0e-a7df49345926 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:54.409221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c5312b8c-e53c-486d-9d0e-a7df49345926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:54.473259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 566/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:54.497753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d44be9-dd4c-4655-9d44-071effc849ec None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:54.595682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d44be9-dd4c-4655-9d44-071effc849ec None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:54.628877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d438c8b-82c2-4916-994d-e079fd09792c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:54.629654 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d438c8b-82c2-4916-994d-e079fd09792c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:54.629966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d438c8b-82c2-4916-994d-e079fd09792c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:54.630243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d438c8b-82c2-4916-994d-e079fd09792c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:54.631417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d438c8b-82c2-4916-994d-e079fd09792c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:54.632902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d438c8b-82c2-4916-994d-e079fd09792c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:54.634473 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 571/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:18:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:54.640151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5312b8c-e53c-486d-9d0e-a7df49345926 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:54.737476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5312b8c-e53c-486d-9d0e-a7df49345926 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:54.758911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:54.772139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ca8af-8766-49ff-b233-83e63b949050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:54.773061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ca8af-8766-49ff-b233-83e63b949050 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:54.773345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ca8af-8766-49ff-b233-83e63b949050 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:54.773623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ca8af-8766-49ff-b233-83e63b949050 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:54.774385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ca8af-8766-49ff-b233-83e63b949050 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:54.775801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ca8af-8766-49ff-b233-83e63b949050 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:54.796205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 567/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:18:54 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:18:56.884430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a18138-3310-4c1e-854f-b15d4a18e734 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:56.884993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a18138-3310-4c1e-854f-b15d4a18e734 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:56.884993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a18138-3310-4c1e-854f-b15d4a18e734 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:56.887198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 572/1139] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:18:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:18:56.892526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b023d5-aa19-4fd9-b024-cad5537f109d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:56.892894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b023d5-aa19-4fd9-b024-cad5537f109d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:56.893091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b023d5-aa19-4fd9-b024-cad5537f109d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:56.910615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-74b023d5-aa19-4fd9-b024-cad5537f109d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:18:56.911778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-74b023d5-aa19-4fd9-b024-cad5537f109d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:18:56.977505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 568/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:18:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:18:57.001038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a18138-3310-4c1e-854f-b15d4a18e734 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:57.068927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:57.105013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a18138-3310-4c1e-854f-b15d4a18e734 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:57.138265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203be604-0c13-4ea5-b820-39f25fda3835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:57.138993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be604-0c13-4ea5-b820-39f25fda3835 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:57.139279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be604-0c13-4ea5-b820-39f25fda3835 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:57.139557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be604-0c13-4ea5-b820-39f25fda3835 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:57.149647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203be604-0c13-4ea5-b820-39f25fda3835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-203be604-0c13-4ea5-b820-39f25fda3835 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:18:57.157820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:18:57.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:18:57.159780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 573/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:18:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:18:57.165177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b023d5-aa19-4fd9-b024-cad5537f109d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:57.259421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b023d5-aa19-4fd9-b024-cad5537f109d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:57.268782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:57.294990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5724fe-cc0a-41ca-bb7d-1cd00b9602a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:57.295635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5724fe-cc0a-41ca-bb7d-1cd00b9602a3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:57.295916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5724fe-cc0a-41ca-bb7d-1cd00b9602a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:57.296190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5724fe-cc0a-41ca-bb7d-1cd00b9602a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:57.296924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5724fe-cc0a-41ca-bb7d-1cd00b9602a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:57.297228 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5724fe-cc0a-41ca-bb7d-1cd00b9602a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:57.298602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5724fe-cc0a-41ca-bb7d-1cd00b9602a3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:57.306341 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 569/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:18:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:57.312785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203be604-0c13-4ea5-b820-39f25fda3835 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:57.412163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203be604-0c13-4ea5-b820-39f25fda3835 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:57.437197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:57.446528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b556ebb-047c-4e8a-8a27-dd0ea7aa29e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:57.447387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556ebb-047c-4e8a-8a27-dd0ea7aa29e6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:57.447667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556ebb-047c-4e8a-8a27-dd0ea7aa29e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:57.447943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556ebb-047c-4e8a-8a27-dd0ea7aa29e6 None admin] PATH_INFO: `/v3/users/74454bb24e1c4c50916d1dd9f7386298` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:57.461241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b556ebb-047c-4e8a-8a27-dd0ea7aa29e6 None admin] RBAC: Authorizing `identity:get_user(user_id=74454bb24e1c4c50916d1dd9f7386298)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:57.462698 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b556ebb-047c-4e8a-8a27-dd0ea7aa29e6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:57.464303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 574/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:18:57 2022] GET /identity/v3/users/74454bb24e1c4c50916d1dd9f7386298 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:57.470053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5724fe-cc0a-41ca-bb7d-1cd00b9602a3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:57.566575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5724fe-cc0a-41ca-bb7d-1cd00b9602a3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:57.599175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef7734-7b08-4644-a1b2-efee804f81d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:57.599816 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef7734-7b08-4644-a1b2-efee804f81d5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:57.600098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef7734-7b08-4644-a1b2-efee804f81d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:57.600373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef7734-7b08-4644-a1b2-efee804f81d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:57.640917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:57.669273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef7734-7b08-4644-a1b2-efee804f81d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:57.670683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef7734-7b08-4644-a1b2-efee804f81d5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:57.769263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 570/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:18:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:18:57.775624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b556ebb-047c-4e8a-8a27-dd0ea7aa29e6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:57.891153 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b556ebb-047c-4e8a-8a27-dd0ea7aa29e6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:57.928206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8bed4-e96b-426c-877c-bc24cb189e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:57.928985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8bed4-e96b-426c-877c-bc24cb189e2d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:57.929271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8bed4-e96b-426c-877c-bc24cb189e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:57.929549 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8bed4-e96b-426c-877c-bc24cb189e2d None admin] PATH_INFO: `/v3/projects/e2c38f369815449b880717b330b90e18` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:57.930767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8bed4-e96b-426c-877c-bc24cb189e2d None admin] RBAC: Authorizing `identity:get_project(project_id=e2c38f369815449b880717b330b90e18)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:57.932222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8bed4-e96b-426c-877c-bc24cb189e2d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:57.934593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 575/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:18:57 2022] GET /identity/v3/projects/e2c38f369815449b880717b330b90e18 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:18:57.944237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef7734-7b08-4644-a1b2-efee804f81d5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:18:58.026817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:18:58.066361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef7734-7b08-4644-a1b2-efee804f81d5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:18:58.102508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89961fb4-2725-401a-94e3-4479fcb7fc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:18:58.103170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961fb4-2725-401a-94e3-4479fcb7fc04 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:18:58.103457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961fb4-2725-401a-94e3-4479fcb7fc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:18:58.103738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961fb4-2725-401a-94e3-4479fcb7fc04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:18:58.105639 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89961fb4-2725-401a-94e3-4479fcb7fc04 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=e2c38f369815449b880717b330b90e18, user.id=74454bb24e1c4c50916d1dd9f7386298) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:18:58.106018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89961fb4-2725-401a-94e3-4479fcb7fc04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:18:58.108068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89961fb4-2725-401a-94e3-4479fcb7fc04 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:18:58.116415 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 571/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:18:57 2022] GET /identity/v3/role_assignments?user.id=74454bb24e1c4c50916d1dd9f7386298&scope.project.id=e2c38f369815449b880717b330b90e18&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:00.141468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd953a-30ba-4a4c-9a5b-99a3904f1a29 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:00.142005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd953a-30ba-4a4c-9a5b-99a3904f1a29 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:00.142098 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd953a-30ba-4a4c-9a5b-99a3904f1a29 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:00.143405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 576/1147] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:19:00.148717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506efb43-6f79-4fee-8664-41bb74c2bd63 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:00.149152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506efb43-6f79-4fee-8664-41bb74c2bd63 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:00.149284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506efb43-6f79-4fee-8664-41bb74c2bd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:00.165467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-506efb43-6f79-4fee-8664-41bb74c2bd63 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:19:00.166653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-506efb43-6f79-4fee-8664-41bb74c2bd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:19:00.230085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 572/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:19:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:19:00.251895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd953a-30ba-4a4c-9a5b-99a3904f1a29 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:00.312936 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:00.359944 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd953a-30ba-4a4c-9a5b-99a3904f1a29 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:00.393732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db535a28-1ffa-4515-817d-9c35058eedcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:00.394534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db535a28-1ffa-4515-817d-9c35058eedcc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:00.394815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db535a28-1ffa-4515-817d-9c35058eedcc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:00.395089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db535a28-1ffa-4515-817d-9c35058eedcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:00.405325 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db535a28-1ffa-4515-817d-9c35058eedcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-db535a28-1ffa-4515-817d-9c35058eedcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:19:00.412861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:19:00.413967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:19:00.414992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 577/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:19:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:19:00.420229 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506efb43-6f79-4fee-8664-41bb74c2bd63 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:00.480632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:00.518612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506efb43-6f79-4fee-8664-41bb74c2bd63 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:00.551453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ed904-dddb-4622-abdd-2e7d582b67e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:00.552102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ed904-dddb-4622-abdd-2e7d582b67e2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:00.552383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ed904-dddb-4622-abdd-2e7d582b67e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:00.552657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ed904-dddb-4622-abdd-2e7d582b67e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:00.553392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ed904-dddb-4622-abdd-2e7d582b67e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:19:00.553698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ed904-dddb-4622-abdd-2e7d582b67e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:00.555071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ed904-dddb-4622-abdd-2e7d582b67e2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:00.563781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 573/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:19:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:00.569589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db535a28-1ffa-4515-817d-9c35058eedcc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:00.670963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db535a28-1ffa-4515-817d-9c35058eedcc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:00.683981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:00.705161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6aa1d-47d5-442f-af5a-d22452403719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:00.706062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6aa1d-47d5-442f-af5a-d22452403719 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:00.706348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6aa1d-47d5-442f-af5a-d22452403719 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:00.706626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6aa1d-47d5-442f-af5a-d22452403719 None admin] PATH_INFO: `/v3/users/74454bb24e1c4c50916d1dd9f7386298` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:00.707810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6aa1d-47d5-442f-af5a-d22452403719 None admin] RBAC: Authorizing `identity:get_user(user_id=74454bb24e1c4c50916d1dd9f7386298)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:00.709246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6aa1d-47d5-442f-af5a-d22452403719 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:00.710933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 578/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:19:00 2022] GET /identity/v3/users/74454bb24e1c4c50916d1dd9f7386298 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:00.717227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ed904-dddb-4622-abdd-2e7d582b67e2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:00.814186 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ed904-dddb-4622-abdd-2e7d582b67e2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:00.837825 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:00.846762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3972f-6989-4b52-9aa3-c1b89ea180a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:00.847618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3972f-6989-4b52-9aa3-c1b89ea180a1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:00.847899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3972f-6989-4b52-9aa3-c1b89ea180a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:00.848173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3972f-6989-4b52-9aa3-c1b89ea180a1 None admin] PATH_INFO: `/v3/projects/e2c38f369815449b880717b330b90e18` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:00.849321 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3972f-6989-4b52-9aa3-c1b89ea180a1 None admin] RBAC: Authorizing `identity:get_project(project_id=e2c38f369815449b880717b330b90e18)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:00.850765 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3972f-6989-4b52-9aa3-c1b89ea180a1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:00.852432 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 574/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:19:00 2022] GET /identity/v3/projects/e2c38f369815449b880717b330b90e18 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:00.859195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6aa1d-47d5-442f-af5a-d22452403719 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:00.957351 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6aa1d-47d5-442f-af5a-d22452403719 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:00.990064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c263a4-20cd-435d-8b91-74da89b3f3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:00.990937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c263a4-20cd-435d-8b91-74da89b3f3b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:00.991221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c263a4-20cd-435d-8b91-74da89b3f3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:00.991497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c263a4-20cd-435d-8b91-74da89b3f3b7 None admin] PATH_INFO: `/v3/projects/e2c38f369815449b880717b330b90e18/users/74454bb24e1c4c50916d1dd9f7386298/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:00.993432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c263a4-20cd-435d-8b91-74da89b3f3b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2c38f369815449b880717b330b90e18, user_id=74454bb24e1c4c50916d1dd9f7386298, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:00.994878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c263a4-20cd-435d-8b91-74da89b3f3b7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:01.007859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 579/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:19:00 2022] PUT /identity/v3/projects/e2c38f369815449b880717b330b90e18/users/74454bb24e1c4c50916d1dd9f7386298/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:19:03.002479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e5961-2ef5-4909-abc7-ce63dde431e5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:03.003012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e5961-2ef5-4909-abc7-ce63dde431e5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:03.003104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e5961-2ef5-4909-abc7-ce63dde431e5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:03.004417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 575/1154] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:19:03.009487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257fed-3f15-4bc9-99ce-702f68dd14c1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:03.009877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257fed-3f15-4bc9-99ce-702f68dd14c1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:03.010081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257fed-3f15-4bc9-99ce-702f68dd14c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:03.026788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4d257fed-3f15-4bc9-99ce-702f68dd14c1 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:19:03.028038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4d257fed-3f15-4bc9-99ce-702f68dd14c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:19:03.062931 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:03.096202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 580/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:19:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:19:03.119762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e5961-2ef5-4909-abc7-ce63dde431e5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:03.219717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e5961-2ef5-4909-abc7-ce63dde431e5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:03.253023 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d49c26-02e2-47b7-baf3-2384446b0d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:03.253742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d49c26-02e2-47b7-baf3-2384446b0d3e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:03.254058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d49c26-02e2-47b7-baf3-2384446b0d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:03.254342 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d49c26-02e2-47b7-baf3-2384446b0d3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:03.264381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d49c26-02e2-47b7-baf3-2384446b0d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d5d49c26-02e2-47b7-baf3-2384446b0d3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:19:03.272026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:19:03.273375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:19:03.274512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 576/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:19:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:19:03.279596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d257fed-3f15-4bc9-99ce-702f68dd14c1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:03.376074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d257fed-3f15-4bc9-99ce-702f68dd14c1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:03.409322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608db564-1bd8-4ee5-b8c3-455c2e56b629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:03.410002 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608db564-1bd8-4ee5-b8c3-455c2e56b629 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:03.410292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608db564-1bd8-4ee5-b8c3-455c2e56b629 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:03.410572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608db564-1bd8-4ee5-b8c3-455c2e56b629 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:03.411321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608db564-1bd8-4ee5-b8c3-455c2e56b629 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:19:03.411632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608db564-1bd8-4ee5-b8c3-455c2e56b629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:03.413001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608db564-1bd8-4ee5-b8c3-455c2e56b629 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:03.421549 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 581/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:19:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:03.427158 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d49c26-02e2-47b7-baf3-2384446b0d3e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:03.481768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:03.526246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d49c26-02e2-47b7-baf3-2384446b0d3e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:03.559671 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d8086-17e9-4183-9e9d-05d0cd4ca44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:03.560512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d8086-17e9-4183-9e9d-05d0cd4ca44b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:03.560791 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d8086-17e9-4183-9e9d-05d0cd4ca44b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:03.561064 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d8086-17e9-4183-9e9d-05d0cd4ca44b None admin] PATH_INFO: `/v3/users/74454bb24e1c4c50916d1dd9f7386298` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:03.562256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d8086-17e9-4183-9e9d-05d0cd4ca44b None admin] RBAC: Authorizing `identity:get_user(user_id=74454bb24e1c4c50916d1dd9f7386298)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:03.563677 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d8086-17e9-4183-9e9d-05d0cd4ca44b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:03.565285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 577/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:19:03 2022] GET /identity/v3/users/74454bb24e1c4c50916d1dd9f7386298 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:03.570915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608db564-1bd8-4ee5-b8c3-455c2e56b629 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:03.601887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:03.669862 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608db564-1bd8-4ee5-b8c3-455c2e56b629 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:03.702181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbb068-d1c3-46ef-bf35-53b15ae89642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:03.702820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbb068-d1c3-46ef-bf35-53b15ae89642 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:03.703103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbb068-d1c3-46ef-bf35-53b15ae89642 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:03.703381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbb068-d1c3-46ef-bf35-53b15ae89642 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:03.768603 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbb068-d1c3-46ef-bf35-53b15ae89642 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:03.770076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbb068-d1c3-46ef-bf35-53b15ae89642 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:03.801668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:03.869503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 582/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:19:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:19:03.876272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d8086-17e9-4183-9e9d-05d0cd4ca44b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:03.975790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d8086-17e9-4183-9e9d-05d0cd4ca44b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:03.998658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:04.010367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4e901-2440-4f62-a1d6-0b0f55c9d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:04.011247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4e901-2440-4f62-a1d6-0b0f55c9d412 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:04.011613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4e901-2440-4f62-a1d6-0b0f55c9d412 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:04.011898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4e901-2440-4f62-a1d6-0b0f55c9d412 None admin] PATH_INFO: `/v3/projects/e2c38f369815449b880717b330b90e18` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:04.013071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4e901-2440-4f62-a1d6-0b0f55c9d412 None admin] RBAC: Authorizing `identity:get_project(project_id=e2c38f369815449b880717b330b90e18)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:04.014547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4e901-2440-4f62-a1d6-0b0f55c9d412 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:04.016238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 578/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:19:03 2022] GET /identity/v3/projects/e2c38f369815449b880717b330b90e18 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:04.022471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbb068-d1c3-46ef-bf35-53b15ae89642 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:04.120269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbb068-d1c3-46ef-bf35-53b15ae89642 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:04.148715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:04.154523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399945b4-e44a-4f46-8599-2599f0ad189b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:04.155176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399945b4-e44a-4f46-8599-2599f0ad189b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:04.155462 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399945b4-e44a-4f46-8599-2599f0ad189b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:04.155743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399945b4-e44a-4f46-8599-2599f0ad189b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:04.156575 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399945b4-e44a-4f46-8599-2599f0ad189b None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=e2c38f369815449b880717b330b90e18, user.id=74454bb24e1c4c50916d1dd9f7386298) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:19:04.156897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399945b4-e44a-4f46-8599-2599f0ad189b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:04.158353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399945b4-e44a-4f46-8599-2599f0ad189b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:04.164107 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 583/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:19:04 2022] GET /identity/v3/role_assignments?user.id=74454bb24e1c4c50916d1dd9f7386298&scope.project.id=e2c38f369815449b880717b330b90e18&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:06.223039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb2573-c255-47eb-9018-d3de70e13c72 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:06.223588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb2573-c255-47eb-9018-d3de70e13c72 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:06.223588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb2573-c255-47eb-9018-d3de70e13c72 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:06.224971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 579/1162] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:19:06.230268 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4aaa6-1cda-457a-87a5-3242a50253fc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:06.230643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4aaa6-1cda-457a-87a5-3242a50253fc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:06.230836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4aaa6-1cda-457a-87a5-3242a50253fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:06.247109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bef4aaa6-1cda-457a-87a5-3242a50253fc None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:19:06.248262 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bef4aaa6-1cda-457a-87a5-3242a50253fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:19:06.314909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 584/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:19:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:19:06.338292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb2573-c255-47eb-9018-d3de70e13c72 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:06.436678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb2573-c255-47eb-9018-d3de70e13c72 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:06.470027 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba952-ab98-4a98-ac8e-2b0829c3897e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:06.470660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba952-ab98-4a98-ac8e-2b0829c3897e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:06.470943 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba952-ab98-4a98-ac8e-2b0829c3897e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:06.471217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba952-ab98-4a98-ac8e-2b0829c3897e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:06.527451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:06.537781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba952-ab98-4a98-ac8e-2b0829c3897e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:06.539246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba952-ab98-4a98-ac8e-2b0829c3897e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:06.634856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 580/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:19:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:19:06.641362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4aaa6-1cda-457a-87a5-3242a50253fc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:06.736709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4aaa6-1cda-457a-87a5-3242a50253fc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:06.761691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:06.770532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfde9be-4dff-4c4a-b79d-c421ad3c9704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:06.771327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfde9be-4dff-4c4a-b79d-c421ad3c9704 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:06.771611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfde9be-4dff-4c4a-b79d-c421ad3c9704 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:06.771887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfde9be-4dff-4c4a-b79d-c421ad3c9704 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:06.778769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfde9be-4dff-4c4a-b79d-c421ad3c9704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:06.780209 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfde9be-4dff-4c4a-b79d-c421ad3c9704 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7bfde9be-4dff-4c4a-b79d-c421ad3c9704 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:19:06.794699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:19:06.795708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:19:06.796618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 585/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 06:19:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:19:06.801820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba952-ab98-4a98-ac8e-2b0829c3897e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:06.889151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:06.899273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba952-ab98-4a98-ac8e-2b0829c3897e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:06.932048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc9d42-74b4-4cee-8efa-f5daa4ed806e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:06.932698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9d42-74b4-4cee-8efa-f5daa4ed806e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:06.933543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9d42-74b4-4cee-8efa-f5daa4ed806e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:06.933822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9d42-74b4-4cee-8efa-f5daa4ed806e None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:06.934571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc9d42-74b4-4cee-8efa-f5daa4ed806e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:19:06.934878 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc9d42-74b4-4cee-8efa-f5daa4ed806e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:06.936221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc9d42-74b4-4cee-8efa-f5daa4ed806e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:06.943110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 581/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 06:19:06 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:08.937485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3897a2-9297-4832-9908-5438cf30277d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:08.938035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3897a2-9297-4832-9908-5438cf30277d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:08.938130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3897a2-9297-4832-9908-5438cf30277d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:08.939434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 586/1167] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:19:08.944542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660f8f3-a8d0-4f3a-890a-e683a04da0f5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:08.944963 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660f8f3-a8d0-4f3a-890a-e683a04da0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:08.945099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660f8f3-a8d0-4f3a-890a-e683a04da0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:08.961734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2660f8f3-a8d0-4f3a-890a-e683a04da0f5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:19:08.962921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2660f8f3-a8d0-4f3a-890a-e683a04da0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:19:09.027831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 582/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:19:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:19:09.050315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3897a2-9297-4832-9908-5438cf30277d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:09.153415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3897a2-9297-4832-9908-5438cf30277d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:09.185555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918864f-62a2-440c-b154-53769e426c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:09.186228 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918864f-62a2-440c-b154-53769e426c7c None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:09.186509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918864f-62a2-440c-b154-53769e426c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:09.186785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918864f-62a2-440c-b154-53769e426c7c None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:09.187581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918864f-62a2-440c-b154-53769e426c7c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:09.188885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918864f-62a2-440c-b154-53769e426c7c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:09.203299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 587/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Thu Jul 21 06:19:09 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:19:11.237663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224631d9-9a07-4086-803b-c1af860c7b93 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:11.238290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224631d9-9a07-4086-803b-c1af860c7b93 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:11.238290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224631d9-9a07-4086-803b-c1af860c7b93 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:11.239613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 583/1170] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:19:11.244645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f04cad-5ea9-4e34-9cfa-f600dce0e9c7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:11.245080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f04cad-5ea9-4e34-9cfa-f600dce0e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:11.245184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f04cad-5ea9-4e34-9cfa-f600dce0e9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:11.262069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-03f04cad-5ea9-4e34-9cfa-f600dce0e9c7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:19:11.263235 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-03f04cad-5ea9-4e34-9cfa-f600dce0e9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:19:11.304454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:11.329909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 588/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:19:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:19:11.352734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224631d9-9a07-4086-803b-c1af860c7b93 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:11.438687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:11.455549 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224631d9-9a07-4086-803b-c1af860c7b93 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:11.488485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dddf57-e55f-405a-9433-57c806c8720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:11.489330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dddf57-e55f-405a-9433-57c806c8720d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:11.489610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dddf57-e55f-405a-9433-57c806c8720d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:11.489883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dddf57-e55f-405a-9433-57c806c8720d None admin] PATH_INFO: `/v3/domains/672e8588773c4d3ea71dcba15ddfd03a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:11.500848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dddf57-e55f-405a-9433-57c806c8720d None admin] RBAC: Authorizing `identity:get_domain(domain_id=672e8588773c4d3ea71dcba15ddfd03a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:11.502304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dddf57-e55f-405a-9433-57c806c8720d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:11.503904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 584/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Jul 21 06:19:11 2022] GET /identity/v3/domains/672e8588773c4d3ea71dcba15ddfd03a => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:11.509673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f04cad-5ea9-4e34-9cfa-f600dce0e9c7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:11.608423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f04cad-5ea9-4e34-9cfa-f600dce0e9c7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:11.641521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0514bf-78c1-4de7-8a37-30ef837885d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:11.642192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0514bf-78c1-4de7-8a37-30ef837885d7 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:11.642480 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0514bf-78c1-4de7-8a37-30ef837885d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:11.642758 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0514bf-78c1-4de7-8a37-30ef837885d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:11.643464 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0514bf-78c1-4de7-8a37-30ef837885d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:11.644890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0514bf-78c1-4de7-8a37-30ef837885d7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:11.670099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 589/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:19:11 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:19:13.706205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa7c84-f648-4aba-833b-425325d8f5ad None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:13.706748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa7c84-f648-4aba-833b-425325d8f5ad None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:13.706748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa7c84-f648-4aba-833b-425325d8f5ad None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:13.708122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 585/1174] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:19:13.713172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da932bd6-1742-482f-a299-ba43fb8918ba None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:13.713557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da932bd6-1742-482f-a299-ba43fb8918ba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:13.713738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da932bd6-1742-482f-a299-ba43fb8918ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:13.730881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-da932bd6-1742-482f-a299-ba43fb8918ba None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:19:13.731860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-da932bd6-1742-482f-a299-ba43fb8918ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:19:13.754350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:13.798768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 590/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:19:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:19:13.823039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa7c84-f648-4aba-833b-425325d8f5ad None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:13.921908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa7c84-f648-4aba-833b-425325d8f5ad None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:13.954438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06a802-a287-4f4c-994e-2baff1b4b2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:13.955306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06a802-a287-4f4c-994e-2baff1b4b2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:13.955588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06a802-a287-4f4c-994e-2baff1b4b2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:13.955862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06a802-a287-4f4c-994e-2baff1b4b2c5 None admin] PATH_INFO: `/v3/domains/672e8588773c4d3ea71dcba15ddfd03a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:13.957082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06a802-a287-4f4c-994e-2baff1b4b2c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=672e8588773c4d3ea71dcba15ddfd03a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:13.958484 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06a802-a287-4f4c-994e-2baff1b4b2c5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:13.960096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 586/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Jul 21 06:19:13 2022] GET /identity/v3/domains/672e8588773c4d3ea71dcba15ddfd03a => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:13.966488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da932bd6-1742-482f-a299-ba43fb8918ba None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:14.064173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da932bd6-1742-482f-a299-ba43fb8918ba None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:14.097476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa1487-1967-4d47-8e88-03a5d21a6b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:14.098171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa1487-1967-4d47-8e88-03a5d21a6b7b None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:14.098459 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa1487-1967-4d47-8e88-03a5d21a6b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:14.098739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa1487-1967-4d47-8e88-03a5d21a6b7b None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:14.099444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa1487-1967-4d47-8e88-03a5d21a6b7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:14.100885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa1487-1967-4d47-8e88-03a5d21a6b7b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:14.120381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 591/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:19:13 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:19:16.215018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042bc89-4f1d-40a8-87f2-30e1dd6b4698 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:16.215532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042bc89-4f1d-40a8-87f2-30e1dd6b4698 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:16.215648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042bc89-4f1d-40a8-87f2-30e1dd6b4698 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:16.217036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 587/1178] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:19:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:19:16.222293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575773d6-3b67-479e-a049-fd22bec55711 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:16.222697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575773d6-3b67-479e-a049-fd22bec55711 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:16.222853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575773d6-3b67-479e-a049-fd22bec55711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:16.239938 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-575773d6-3b67-479e-a049-fd22bec55711 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:19:16.241112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-575773d6-3b67-479e-a049-fd22bec55711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:19:16.250254 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:16.308095 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 592/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:19:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:19:16.330775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042bc89-4f1d-40a8-87f2-30e1dd6b4698 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:16.345752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:16.433965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042bc89-4f1d-40a8-87f2-30e1dd6b4698 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:16.467378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c941b5-5e17-40d3-9670-eeafb21f41da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:16.468162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c941b5-5e17-40d3-9670-eeafb21f41da None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:16.468450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c941b5-5e17-40d3-9670-eeafb21f41da None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:16.468729 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c941b5-5e17-40d3-9670-eeafb21f41da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:16.478465 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c941b5-5e17-40d3-9670-eeafb21f41da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-08c941b5-5e17-40d3-9670-eeafb21f41da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:19:16.486272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:19:16.487427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:19:16.488489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 588/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:19:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:19:16.493412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575773d6-3b67-479e-a049-fd22bec55711 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:16.590522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575773d6-3b67-479e-a049-fd22bec55711 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:16.624996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4438b7-bc7f-4ec5-af56-081eed564d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:16.625651 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4438b7-bc7f-4ec5-af56-081eed564d7a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:16.625962 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4438b7-bc7f-4ec5-af56-081eed564d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:16.626243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4438b7-bc7f-4ec5-af56-081eed564d7a None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:16.626991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4438b7-bc7f-4ec5-af56-081eed564d7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:19:16.627302 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4438b7-bc7f-4ec5-af56-081eed564d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:16.628679 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4438b7-bc7f-4ec5-af56-081eed564d7a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:16.636876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 593/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:19:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:16.642639 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c941b5-5e17-40d3-9670-eeafb21f41da None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:16.711560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:16.740596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c941b5-5e17-40d3-9670-eeafb21f41da None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:16.773471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d2a9b-b631-4333-94df-9cc2e2178ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:16.774363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d2a9b-b631-4333-94df-9cc2e2178ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:16.774670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d2a9b-b631-4333-94df-9cc2e2178ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:16.774947 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d2a9b-b631-4333-94df-9cc2e2178ce5 None admin] PATH_INFO: `/v3/users/b6cfab4cddf049828000374e7860425b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:16.787630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d2a9b-b631-4333-94df-9cc2e2178ce5 None admin] RBAC: Authorizing `identity:get_user(user_id=b6cfab4cddf049828000374e7860425b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:16.789059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d2a9b-b631-4333-94df-9cc2e2178ce5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:16.790727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 589/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:19:16 2022] GET /identity/v3/users/b6cfab4cddf049828000374e7860425b => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:16.796926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4438b7-bc7f-4ec5-af56-081eed564d7a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:16.808100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:16.898656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4438b7-bc7f-4ec5-af56-081eed564d7a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:16.932153 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec8a01-ee37-4d33-87d8-b652cc58c082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:16.932790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec8a01-ee37-4d33-87d8-b652cc58c082 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:16.933069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec8a01-ee37-4d33-87d8-b652cc58c082 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:16.933807 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec8a01-ee37-4d33-87d8-b652cc58c082 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:17.000823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec8a01-ee37-4d33-87d8-b652cc58c082 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:17.002294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec8a01-ee37-4d33-87d8-b652cc58c082 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:17.009722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:17.101390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 594/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:19:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:19:17.107851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d2a9b-b631-4333-94df-9cc2e2178ce5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:17.204392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d2a9b-b631-4333-94df-9cc2e2178ce5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:17.231806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:17.237796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d485de7-24a9-4cad-a771-46ad87166856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:17.238677 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d485de7-24a9-4cad-a771-46ad87166856 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:17.238966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d485de7-24a9-4cad-a771-46ad87166856 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:17.239244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d485de7-24a9-4cad-a771-46ad87166856 None admin] PATH_INFO: `/v3/projects/7426c7ad07d54e7e9a891e0ec4c9b0d7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:17.240416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d485de7-24a9-4cad-a771-46ad87166856 None admin] RBAC: Authorizing `identity:get_project(project_id=7426c7ad07d54e7e9a891e0ec4c9b0d7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:17.241828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d485de7-24a9-4cad-a771-46ad87166856 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:17.243527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 590/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:19:17 2022] GET /identity/v3/projects/7426c7ad07d54e7e9a891e0ec4c9b0d7 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:17.249692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec8a01-ee37-4d33-87d8-b652cc58c082 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:17.347439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec8a01-ee37-4d33-87d8-b652cc58c082 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:17.354483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:17.380625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ae0ff-1055-4ff1-87bd-33b5ea9a5d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:17.381269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ae0ff-1055-4ff1-87bd-33b5ea9a5d37 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:17.381553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ae0ff-1055-4ff1-87bd-33b5ea9a5d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:17.381831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ae0ff-1055-4ff1-87bd-33b5ea9a5d37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:17.382698 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ae0ff-1055-4ff1-87bd-33b5ea9a5d37 None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=7426c7ad07d54e7e9a891e0ec4c9b0d7, user.id=b6cfab4cddf049828000374e7860425b) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:19:17.383012 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ae0ff-1055-4ff1-87bd-33b5ea9a5d37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:17.384431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ae0ff-1055-4ff1-87bd-33b5ea9a5d37 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:17.390469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 595/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:19:17 2022] GET /identity/v3/role_assignments?user.id=b6cfab4cddf049828000374e7860425b&scope.project.id=7426c7ad07d54e7e9a891e0ec4c9b0d7&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:19.368094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5798e0-7629-4cd2-b851-3a85c9e0bb95 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:19.368605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5798e0-7629-4cd2-b851-3a85c9e0bb95 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:19.368605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5798e0-7629-4cd2-b851-3a85c9e0bb95 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:19.369959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 591/1186] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:19:19.375633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1166-d09f-4755-8011-e600d9ad9968 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:19.376036 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1166-d09f-4755-8011-e600d9ad9968 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:19.376199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1166-d09f-4755-8011-e600d9ad9968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:19.393301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-412d1166-d09f-4755-8011-e600d9ad9968 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:19:19.394510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-412d1166-d09f-4755-8011-e600d9ad9968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:19:19.460204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 596/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:19:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:19:19.482825 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5798e0-7629-4cd2-b851-3a85c9e0bb95 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:19.582479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5798e0-7629-4cd2-b851-3a85c9e0bb95 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:19.615118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95d9ee-ca30-489e-b489-c6481ba2b8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:19.615900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95d9ee-ca30-489e-b489-c6481ba2b8be None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:19.616184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95d9ee-ca30-489e-b489-c6481ba2b8be None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:19.616462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95d9ee-ca30-489e-b489-c6481ba2b8be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:19.627197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95d9ee-ca30-489e-b489-c6481ba2b8be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6a95d9ee-ca30-489e-b489-c6481ba2b8be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:19:19.634741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:19:19.635910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:19:19.636995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 592/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:19:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:19:19.641918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d1166-d09f-4755-8011-e600d9ad9968 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:19.736904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:19.739798 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d1166-d09f-4755-8011-e600d9ad9968 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:19.774217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d44fe-e872-4981-9dac-a5fd391a4b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:19.774881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d44fe-e872-4981-9dac-a5fd391a4b86 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:19.775171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d44fe-e872-4981-9dac-a5fd391a4b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:19.775450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d44fe-e872-4981-9dac-a5fd391a4b86 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:19.776199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d44fe-e872-4981-9dac-a5fd391a4b86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:19:19.776506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d44fe-e872-4981-9dac-a5fd391a4b86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:19.778526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d44fe-e872-4981-9dac-a5fd391a4b86 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:19.786446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 597/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:19:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:19.792388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95d9ee-ca30-489e-b489-c6481ba2b8be None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:19.850591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:19.892327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95d9ee-ca30-489e-b489-c6481ba2b8be None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:19.924460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebc036-c454-4e91-9961-49682ab93c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:19.925383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebc036-c454-4e91-9961-49682ab93c80 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:19.925672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebc036-c454-4e91-9961-49682ab93c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:19.925978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebc036-c454-4e91-9961-49682ab93c80 None admin] PATH_INFO: `/v3/users/b6cfab4cddf049828000374e7860425b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:19.927148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebc036-c454-4e91-9961-49682ab93c80 None admin] RBAC: Authorizing `identity:get_user(user_id=b6cfab4cddf049828000374e7860425b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:19.928604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebc036-c454-4e91-9961-49682ab93c80 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:19.930236 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 593/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:19:19 2022] GET /identity/v3/users/b6cfab4cddf049828000374e7860425b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:19.937014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d44fe-e872-4981-9dac-a5fd391a4b86 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:20.034531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d44fe-e872-4981-9dac-a5fd391a4b86 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:20.068029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbff129-b7e5-4ed5-ab43-5a164f291750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:20.068894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbff129-b7e5-4ed5-ab43-5a164f291750 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:20.069177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbff129-b7e5-4ed5-ab43-5a164f291750 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:20.069457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbff129-b7e5-4ed5-ab43-5a164f291750 None admin] PATH_INFO: `/v3/projects/7426c7ad07d54e7e9a891e0ec4c9b0d7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:20.070636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbff129-b7e5-4ed5-ab43-5a164f291750 None admin] RBAC: Authorizing `identity:get_project(project_id=7426c7ad07d54e7e9a891e0ec4c9b0d7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:20.072075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbff129-b7e5-4ed5-ab43-5a164f291750 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:20.073758 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 598/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:19:19 2022] GET /identity/v3/projects/7426c7ad07d54e7e9a891e0ec4c9b0d7 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:20.080405 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebc036-c454-4e91-9961-49682ab93c80 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:20.176507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebc036-c454-4e91-9961-49682ab93c80 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:20.201285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:20.210265 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c97406-6afa-4717-8d12-5f16ef2bcd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:20.211151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97406-6afa-4717-8d12-5f16ef2bcd15 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:20.211437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97406-6afa-4717-8d12-5f16ef2bcd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:20.211716 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97406-6afa-4717-8d12-5f16ef2bcd15 None admin] PATH_INFO: `/v3/projects/7426c7ad07d54e7e9a891e0ec4c9b0d7/users/b6cfab4cddf049828000374e7860425b/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:20.213664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c97406-6afa-4717-8d12-5f16ef2bcd15 None admin] RBAC: Authorizing `identity:create_grant(project_id=7426c7ad07d54e7e9a891e0ec4c9b0d7, user_id=b6cfab4cddf049828000374e7860425b, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:20.215249 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c97406-6afa-4717-8d12-5f16ef2bcd15 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:20.227259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 594/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:19:20 2022] PUT /identity/v3/projects/7426c7ad07d54e7e9a891e0ec4c9b0d7/users/b6cfab4cddf049828000374e7860425b/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:19:22.229720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46a3fc-202f-4c5e-874a-11992e8827cc None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:22.230515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46a3fc-202f-4c5e-874a-11992e8827cc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:22.230515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46a3fc-202f-4c5e-874a-11992e8827cc None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:22.231799 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 599/1193] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:19:22.236880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3b27f-9e31-4119-9c38-930875a1cfb2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:22.237253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3b27f-9e31-4119-9c38-930875a1cfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:22.237452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3b27f-9e31-4119-9c38-930875a1cfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:22.254593 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-25d3b27f-9e31-4119-9c38-930875a1cfb2 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:19:22.255768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-25d3b27f-9e31-4119-9c38-930875a1cfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:19:22.321234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 595/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:19:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:19:22.344960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46a3fc-202f-4c5e-874a-11992e8827cc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:22.376051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:22.450650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46a3fc-202f-4c5e-874a-11992e8827cc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:22.483754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609db9d-d4e9-4c42-99c9-6ac1cded9c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:22.484539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609db9d-d4e9-4c42-99c9-6ac1cded9c65 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:22.484822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609db9d-d4e9-4c42-99c9-6ac1cded9c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:22.485099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609db9d-d4e9-4c42-99c9-6ac1cded9c65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:22.494999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609db9d-d4e9-4c42-99c9-6ac1cded9c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d609db9d-d4e9-4c42-99c9-6ac1cded9c65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:19:22.503306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:19:22.504389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:19:22.505416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 600/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:19:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:19:22.510524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3b27f-9e31-4119-9c38-930875a1cfb2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:22.606890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3b27f-9e31-4119-9c38-930875a1cfb2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:22.622891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:22.640952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211dfe-7e3f-46f5-8768-17fd55c856e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:22.641612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211dfe-7e3f-46f5-8768-17fd55c856e8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:22.641898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211dfe-7e3f-46f5-8768-17fd55c856e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:22.642207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211dfe-7e3f-46f5-8768-17fd55c856e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:22.642955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211dfe-7e3f-46f5-8768-17fd55c856e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:19:22.643266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211dfe-7e3f-46f5-8768-17fd55c856e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:22.644628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211dfe-7e3f-46f5-8768-17fd55c856e8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:22.653310 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 596/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:19:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:22.658919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609db9d-d4e9-4c42-99c9-6ac1cded9c65 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:22.748314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:22.758306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609db9d-d4e9-4c42-99c9-6ac1cded9c65 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:22.790763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad19d17-9ce6-4378-afab-916705930f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:22.791620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad19d17-9ce6-4378-afab-916705930f8d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:22.791904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad19d17-9ce6-4378-afab-916705930f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:22.792185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad19d17-9ce6-4378-afab-916705930f8d None admin] PATH_INFO: `/v3/users/b6cfab4cddf049828000374e7860425b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:22.793990 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad19d17-9ce6-4378-afab-916705930f8d None admin] RBAC: Authorizing `identity:get_user(user_id=b6cfab4cddf049828000374e7860425b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:22.795415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad19d17-9ce6-4378-afab-916705930f8d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:22.797000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 601/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 06:19:22 2022] GET /identity/v3/users/b6cfab4cddf049828000374e7860425b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:22.803184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211dfe-7e3f-46f5-8768-17fd55c856e8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:22.899644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211dfe-7e3f-46f5-8768-17fd55c856e8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:22.932536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1e90c-2e3d-4caa-bd69-1c7e2fff36c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:22.933172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1e90c-2e3d-4caa-bd69-1c7e2fff36c6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:22.933802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1e90c-2e3d-4caa-bd69-1c7e2fff36c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:22.933916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1e90c-2e3d-4caa-bd69-1c7e2fff36c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:22.977926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:22.999728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1e90c-2e3d-4caa-bd69-1c7e2fff36c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:23.001166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1e90c-2e3d-4caa-bd69-1c7e2fff36c6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:23.096570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 597/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:19:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 06:19:23.102412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad19d17-9ce6-4378-afab-916705930f8d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:23.199550 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad19d17-9ce6-4378-afab-916705930f8d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:23.231800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443b425-9512-4721-a151-bab6229c3946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:23.232645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443b425-9512-4721-a151-bab6229c3946 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:23.232928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443b425-9512-4721-a151-bab6229c3946 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:23.233206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443b425-9512-4721-a151-bab6229c3946 None admin] PATH_INFO: `/v3/projects/7426c7ad07d54e7e9a891e0ec4c9b0d7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:23.234401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443b425-9512-4721-a151-bab6229c3946 None admin] RBAC: Authorizing `identity:get_project(project_id=7426c7ad07d54e7e9a891e0ec4c9b0d7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:23.235837 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443b425-9512-4721-a151-bab6229c3946 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:23.237529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 602/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 06:19:23 2022] GET /identity/v3/projects/7426c7ad07d54e7e9a891e0ec4c9b0d7 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:19:23.243423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1e90c-2e3d-4caa-bd69-1c7e2fff36c6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:19:23.318273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:23.339581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1e90c-2e3d-4caa-bd69-1c7e2fff36c6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:19:23.373536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce1eda-4a14-4c1c-b791-3eb697f443fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:19:23.374207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce1eda-4a14-4c1c-b791-3eb697f443fe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:19:23.374497 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce1eda-4a14-4c1c-b791-3eb697f443fe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:19:23.374776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce1eda-4a14-4c1c-b791-3eb697f443fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:19:23.375600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce1eda-4a14-4c1c-b791-3eb697f443fe None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=7426c7ad07d54e7e9a891e0ec4c9b0d7, user.id=b6cfab4cddf049828000374e7860425b) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:19:23.375911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce1eda-4a14-4c1c-b791-3eb697f443fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:19:23.377311 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce1eda-4a14-4c1c-b791-3eb697f443fe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:19:23.382980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 598/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:19:23 2022] GET /identity/v3/role_assignments?user.id=b6cfab4cddf049828000374e7860425b&scope.project.id=7426c7ad07d54e7e9a891e0ec4c9b0d7&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:19:33.227617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=87 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:19:33.380486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=29 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:18.882585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dad5e1-7cfd-45b6-adf6-ab516f80aeca None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:18.883201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dad5e1-7cfd-45b6-adf6-ab516f80aeca None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:18.883201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dad5e1-7cfd-45b6-adf6-ab516f80aeca None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:18.884776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 603/1201] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:20:18.890390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b94d67-ff00-4a59-b080-7b6dbdc85d90 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:18.890915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b94d67-ff00-4a59-b080-7b6dbdc85d90 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:18.891024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b94d67-ff00-4a59-b080-7b6dbdc85d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:18.908649 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d8b94d67-ff00-4a59-b080-7b6dbdc85d90 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:20:18.909839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d8b94d67-ff00-4a59-b080-7b6dbdc85d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:20:18.999971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 599/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:20:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:20:19.025368 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dad5e1-7cfd-45b6-adf6-ab516f80aeca None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:19.181044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dad5e1-7cfd-45b6-adf6-ab516f80aeca None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:19.226720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a37559-37b8-4ada-ac90-b760477358c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:19.227626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a37559-37b8-4ada-ac90-b760477358c1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:19.227926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a37559-37b8-4ada-ac90-b760477358c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:19.228276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a37559-37b8-4ada-ac90-b760477358c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:19.244164 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a37559-37b8-4ada-ac90-b760477358c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:19.245723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a37559-37b8-4ada-ac90-b760477358c1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d7a37559-37b8-4ada-ac90-b760477358c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:19.268555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:19.269699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:19.271010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:19.271010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:19.271010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:19.271010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:19.271010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 604/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:19.276612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b94d67-ff00-4a59-b080-7b6dbdc85d90 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:19.416438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b94d67-ff00-4a59-b080-7b6dbdc85d90 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:19.455608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:19.463903 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e4d65-4a80-48fc-96d7-63a76e5b6b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:19.464804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e4d65-4a80-48fc-96d7-63a76e5b6b31 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:19.465129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e4d65-4a80-48fc-96d7-63a76e5b6b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:19.465432 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e4d65-4a80-48fc-96d7-63a76e5b6b31 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:19.466351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e4d65-4a80-48fc-96d7-63a76e5b6b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:19.467482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e4d65-4a80-48fc-96d7-63a76e5b6b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:19.469171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e4d65-4a80-48fc-96d7-63a76e5b6b31 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:19.484179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 600/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:19.492602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a37559-37b8-4ada-ac90-b760477358c1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:19.566347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:19.634418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a37559-37b8-4ada-ac90-b760477358c1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:19.676563 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfeaf0-9ade-46eb-8a5b-004ae219837e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:19.677491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfeaf0-9ade-46eb-8a5b-004ae219837e None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:19.677886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfeaf0-9ade-46eb-8a5b-004ae219837e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:19.678405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfeaf0-9ade-46eb-8a5b-004ae219837e None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:19.679266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfeaf0-9ade-46eb-8a5b-004ae219837e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:19.680695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfeaf0-9ade-46eb-8a5b-004ae219837e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:19.714450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 605/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 06:20:19 2022] POST /identity/v3/users => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:20:22.091017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e034b4a-67a2-4995-8d41-88b3b1b2997b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:22.091622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e034b4a-67a2-4995-8d41-88b3b1b2997b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:22.091622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e034b4a-67a2-4995-8d41-88b3b1b2997b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:22.093914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 601/1206] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:20:22.099852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374fe69-0bdb-445e-980f-cb33feb862a2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:22.100341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374fe69-0bdb-445e-980f-cb33feb862a2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:22.100441 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374fe69-0bdb-445e-980f-cb33feb862a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:22.118313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8374fe69-0bdb-445e-980f-cb33feb862a2 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:20:22.119516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8374fe69-0bdb-445e-980f-cb33feb862a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:20:22.206520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 606/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:20:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:20:22.234574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e034b4a-67a2-4995-8d41-88b3b1b2997b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:22.353779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e034b4a-67a2-4995-8d41-88b3b1b2997b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:22.396956 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b68a37-bf02-436e-a5f0-7d49a9ee8b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:22.397814 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b68a37-bf02-436e-a5f0-7d49a9ee8b74 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:22.398170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b68a37-bf02-436e-a5f0-7d49a9ee8b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:22.398420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b68a37-bf02-436e-a5f0-7d49a9ee8b74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:22.412181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b68a37-bf02-436e-a5f0-7d49a9ee8b74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-69b68a37-bf02-436e-a5f0-7d49a9ee8b74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:20:22.420210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:22.422051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:22.422905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 602/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:20:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:22.428126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374fe69-0bdb-445e-980f-cb33feb862a2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:22.473077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:22.552147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374fe69-0bdb-445e-980f-cb33feb862a2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:22.593290 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cef0c-5e7c-4a53-9db5-3eadbe70f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:22.594246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cef0c-5e7c-4a53-9db5-3eadbe70f1cb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:22.594567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cef0c-5e7c-4a53-9db5-3eadbe70f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:22.594887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cef0c-5e7c-4a53-9db5-3eadbe70f1cb None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:22.595828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cef0c-5e7c-4a53-9db5-3eadbe70f1cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:22.596166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cef0c-5e7c-4a53-9db5-3eadbe70f1cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:22.597774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cef0c-5e7c-4a53-9db5-3eadbe70f1cb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:22.608299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 607/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:20:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:22.614275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b68a37-bf02-436e-a5f0-7d49a9ee8b74 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:22.687466 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:22.749535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b68a37-bf02-436e-a5f0-7d49a9ee8b74 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:22.788925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe841a94-341e-4404-bf26-375bdff7c6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:22.789757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe841a94-341e-4404-bf26-375bdff7c6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:22.790199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe841a94-341e-4404-bf26-375bdff7c6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:22.790446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe841a94-341e-4404-bf26-375bdff7c6e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:22.803089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe841a94-341e-4404-bf26-375bdff7c6e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:22.804918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe841a94-341e-4404-bf26-375bdff7c6e8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fe841a94-341e-4404-bf26-375bdff7c6e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:22.830780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:22.832176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:22.833488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:22.833488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:22.833488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:22.833488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:22.838122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 603/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:22.838748 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cef0c-5e7c-4a53-9db5-3eadbe70f1cb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:22.948742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:22.952496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cef0c-5e7c-4a53-9db5-3eadbe70f1cb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:22.989634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f3401-6320-436a-b2fa-67d61b237c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:22.990346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f3401-6320-436a-b2fa-67d61b237c1a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:22.990631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f3401-6320-436a-b2fa-67d61b237c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:22.990908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f3401-6320-436a-b2fa-67d61b237c1a None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:22.991636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f3401-6320-436a-b2fa-67d61b237c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:22.991948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f3401-6320-436a-b2fa-67d61b237c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:22.993663 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f3401-6320-436a-b2fa-67d61b237c1a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:23.009650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 608/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:23.016753 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe841a94-341e-4404-bf26-375bdff7c6e8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:23.147960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe841a94-341e-4404-bf26-375bdff7c6e8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:23.149658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:23.190149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7a88e-2ac4-4bc4-bea2-d8ddd3ed2745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:23.191088 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a88e-2ac4-4bc4-bea2-d8ddd3ed2745 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:23.191422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a88e-2ac4-4bc4-bea2-d8ddd3ed2745 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:23.191728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a88e-2ac4-4bc4-bea2-d8ddd3ed2745 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:23.200438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7a88e-2ac4-4bc4-bea2-d8ddd3ed2745 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:23.202762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7a88e-2ac4-4bc4-bea2-d8ddd3ed2745 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b3a7a88e-2ac4-4bc4-bea2-d8ddd3ed2745 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:20:23.218762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:23.220183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:23.221163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:23.221163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:23.221163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:20:23.221163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:20:23.221163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:20:23.221163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:20:23.221163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:20:23.221163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:20:23.221163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:23.221163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:23.221163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 604/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:23.227012 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f3401-6320-436a-b2fa-67d61b237c1a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:23.342294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f3401-6320-436a-b2fa-67d61b237c1a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:23.381016 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745bb8-175a-4aea-a058-49dfc5d2a0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:23.381728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745bb8-175a-4aea-a058-49dfc5d2a0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:23.382073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745bb8-175a-4aea-a058-49dfc5d2a0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:23.382349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745bb8-175a-4aea-a058-49dfc5d2a0a5 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:23.383247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff745bb8-175a-4aea-a058-49dfc5d2a0a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:23.383558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff745bb8-175a-4aea-a058-49dfc5d2a0a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:23.385099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff745bb8-175a-4aea-a058-49dfc5d2a0a5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:23.411514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 609/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 06:20:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:23.412965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7a88e-2ac4-4bc4-bea2-d8ddd3ed2745 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:23.542185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7a88e-2ac4-4bc4-bea2-d8ddd3ed2745 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:23.589190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963049cb-2a6d-45f1-ab43-eda6448fdff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:23.589954 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963049cb-2a6d-45f1-ab43-eda6448fdff3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:23.590443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963049cb-2a6d-45f1-ab43-eda6448fdff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:23.590759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963049cb-2a6d-45f1-ab43-eda6448fdff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:23.624176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:23.668916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963049cb-2a6d-45f1-ab43-eda6448fdff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:23.671482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963049cb-2a6d-45f1-ab43-eda6448fdff3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:23.785732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 605/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:20:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:20:23.791894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745bb8-175a-4aea-a058-49dfc5d2a0a5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:23.836653 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:23.900701 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745bb8-175a-4aea-a058-49dfc5d2a0a5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:23.941838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdf942-8368-4f5c-95b4-6536bc1f86cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:23.942658 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf942-8368-4f5c-95b4-6536bc1f86cb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:23.942944 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf942-8368-4f5c-95b4-6536bc1f86cb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:23.943221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf942-8368-4f5c-95b4-6536bc1f86cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:23.954536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdf942-8368-4f5c-95b4-6536bc1f86cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:23.955999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdf942-8368-4f5c-95b4-6536bc1f86cb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-63bdf942-8368-4f5c-95b4-6536bc1f86cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:23.975768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:23.976900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:23.982373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:23.982373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:23.982373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:23.982373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:23.982373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 610/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:20:23.982900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963049cb-2a6d-45f1-ab43-eda6448fdff3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:24.047416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:24.089038 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963049cb-2a6d-45f1-ab43-eda6448fdff3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:24.125055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4deee-b659-467f-b288-dbcc7a8919cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:24.125731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4deee-b659-467f-b288-dbcc7a8919cf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:24.126048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4deee-b659-467f-b288-dbcc7a8919cf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:24.126332 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4deee-b659-467f-b288-dbcc7a8919cf None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:24.127073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4deee-b659-467f-b288-dbcc7a8919cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:24.127387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4deee-b659-467f-b288-dbcc7a8919cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:24.128754 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4deee-b659-467f-b288-dbcc7a8919cf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:24.141380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 606/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:24.147459 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdf942-8368-4f5c-95b4-6536bc1f86cb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:24.224398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:24.257720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdf942-8368-4f5c-95b4-6536bc1f86cb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:24.298449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dabd7e-9657-4544-a90d-9a7bc9913fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:24.299272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabd7e-9657-4544-a90d-9a7bc9913fea None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:24.299558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabd7e-9657-4544-a90d-9a7bc9913fea None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:24.299886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabd7e-9657-4544-a90d-9a7bc9913fea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:24.308628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dabd7e-9657-4544-a90d-9a7bc9913fea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:24.310200 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dabd7e-9657-4544-a90d-9a7bc9913fea None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d0dabd7e-9657-4544-a90d-9a7bc9913fea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:20:24.325213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:24.326406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:24.332335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:20:24.332335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:20:24.332335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:20:24.332335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:20:24.332335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:20:24.332335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:20:24.332335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:24.332335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:24.332335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 611/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:20:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:24.332936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4deee-b659-467f-b288-dbcc7a8919cf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:24.441769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4deee-b659-467f-b288-dbcc7a8919cf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:24.468016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:24.481301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf0308-77f6-4362-8357-6eb6f1c01941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:24.482306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0308-77f6-4362-8357-6eb6f1c01941 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:24.482531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0308-77f6-4362-8357-6eb6f1c01941 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:24.482953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0308-77f6-4362-8357-6eb6f1c01941 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:24.483854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf0308-77f6-4362-8357-6eb6f1c01941 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:24.484270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf0308-77f6-4362-8357-6eb6f1c01941 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:24.485798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf0308-77f6-4362-8357-6eb6f1c01941 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:24.505981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 607/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:20:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:24.508125 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dabd7e-9657-4544-a90d-9a7bc9913fea None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:24.625896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dabd7e-9657-4544-a90d-9a7bc9913fea None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:24.658802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:24.667101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe4c24-1ca9-41e0-8749-47559d63d976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:24.667976 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe4c24-1ca9-41e0-8749-47559d63d976 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:24.668359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe4c24-1ca9-41e0-8749-47559d63d976 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:24.668776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe4c24-1ca9-41e0-8749-47559d63d976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:24.669808 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe4c24-1ca9-41e0-8749-47559d63d976 None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=1da2d85eb18049f196aeefc51d3f1e0a) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:24.670421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe4c24-1ca9-41e0-8749-47559d63d976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:24.672583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe4c24-1ca9-41e0-8749-47559d63d976 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:24.682407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 612/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:20:24 2022] GET /identity/v3/role_assignments?user.id=1da2d85eb18049f196aeefc51d3f1e0a&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:20:26.866169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddc606-4bc3-4d25-b00e-71bbcf6b792a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:26.866718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddc606-4bc3-4d25-b00e-71bbcf6b792a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:26.866718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddc606-4bc3-4d25-b00e-71bbcf6b792a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:26.868109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 608/1220] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:20:26.873243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edff18-d456-4e21-a692-9bfe75d64288 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:26.873702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edff18-d456-4e21-a692-9bfe75d64288 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:26.873855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edff18-d456-4e21-a692-9bfe75d64288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:26.890862 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-34edff18-d456-4e21-a692-9bfe75d64288 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:20:26.892045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-34edff18-d456-4e21-a692-9bfe75d64288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:20:26.962296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 613/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:20:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:20:26.989372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddc606-4bc3-4d25-b00e-71bbcf6b792a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:27.102748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddc606-4bc3-4d25-b00e-71bbcf6b792a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:27.140308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64f5b5-98cf-479c-b7c6-69a0c044dc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:27.141064 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64f5b5-98cf-479c-b7c6-69a0c044dc2b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:27.141347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64f5b5-98cf-479c-b7c6-69a0c044dc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:27.141623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64f5b5-98cf-479c-b7c6-69a0c044dc2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:27.153752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64f5b5-98cf-479c-b7c6-69a0c044dc2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0f64f5b5-98cf-479c-b7c6-69a0c044dc2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:20:27.161761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:27.162783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:27.163656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 609/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:20:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:27.169093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edff18-d456-4e21-a692-9bfe75d64288 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:27.291045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edff18-d456-4e21-a692-9bfe75d64288 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:27.320015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:27.332145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae2ece-e713-4344-800f-746858e9db83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:27.332821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae2ece-e713-4344-800f-746858e9db83 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:27.333100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae2ece-e713-4344-800f-746858e9db83 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:27.333377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae2ece-e713-4344-800f-746858e9db83 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:27.334155 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae2ece-e713-4344-800f-746858e9db83 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:27.334465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae2ece-e713-4344-800f-746858e9db83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:27.335829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae2ece-e713-4344-800f-746858e9db83 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:27.345631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 614/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:20:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:27.351564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64f5b5-98cf-479c-b7c6-69a0c044dc2b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:27.398648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:27.468774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64f5b5-98cf-479c-b7c6-69a0c044dc2b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:27.504343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d08ce9-342d-4472-a010-cefb67d67104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:27.505142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d08ce9-342d-4472-a010-cefb67d67104 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:27.505427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d08ce9-342d-4472-a010-cefb67d67104 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:27.505704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d08ce9-342d-4472-a010-cefb67d67104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:27.518572 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d08ce9-342d-4472-a010-cefb67d67104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:27.520018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d08ce9-342d-4472-a010-cefb67d67104 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e4d08ce9-342d-4472-a010-cefb67d67104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:27.538111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:27.539218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:27.540273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:27.540273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:27.540273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:27.540273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:27.540273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 610/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:27.545669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae2ece-e713-4344-800f-746858e9db83 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:27.659643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae2ece-e713-4344-800f-746858e9db83 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:27.696457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648e4a8-22aa-40cd-941f-acbd6dcc4829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:27.697149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e4a8-22aa-40cd-941f-acbd6dcc4829 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:27.697450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e4a8-22aa-40cd-941f-acbd6dcc4829 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:27.697796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e4a8-22aa-40cd-941f-acbd6dcc4829 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:27.698570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648e4a8-22aa-40cd-941f-acbd6dcc4829 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:27.698947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648e4a8-22aa-40cd-941f-acbd6dcc4829 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:27.700435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648e4a8-22aa-40cd-941f-acbd6dcc4829 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:27.715860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 615/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:20:27.721784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d08ce9-342d-4472-a010-cefb67d67104 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:27.824521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:27.833821 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d08ce9-342d-4472-a010-cefb67d67104 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:27.870085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b1e98-1581-487f-91d3-89c38469feeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:27.871033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b1e98-1581-487f-91d3-89c38469feeb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:27.871323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b1e98-1581-487f-91d3-89c38469feeb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:27.871599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b1e98-1581-487f-91d3-89c38469feeb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:27.879688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b1e98-1581-487f-91d3-89c38469feeb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:27.881211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b1e98-1581-487f-91d3-89c38469feeb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ec8b1e98-1581-487f-91d3-89c38469feeb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:20:27.896345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:27.897530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:27.898675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:27.898675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:27.898675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:20:27.898675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:20:27.898675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:20:27.898675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:20:27.898675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:20:27.898675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:20:27.898675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:27.898675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:27.898675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 611/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:27.903676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648e4a8-22aa-40cd-941f-acbd6dcc4829 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:27.958342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:28.017278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648e4a8-22aa-40cd-941f-acbd6dcc4829 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:28.062608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff65cd5-be05-4a0f-a1c1-bce998c4c30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:28.063294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff65cd5-be05-4a0f-a1c1-bce998c4c30f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:28.063579 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff65cd5-be05-4a0f-a1c1-bce998c4c30f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:28.063861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff65cd5-be05-4a0f-a1c1-bce998c4c30f None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:28.064717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff65cd5-be05-4a0f-a1c1-bce998c4c30f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:28.065029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff65cd5-be05-4a0f-a1c1-bce998c4c30f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:28.066508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff65cd5-be05-4a0f-a1c1-bce998c4c30f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:28.084478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 616/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 06:20:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:28.090285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b1e98-1581-487f-91d3-89c38469feeb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:28.201738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b1e98-1581-487f-91d3-89c38469feeb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:28.238409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955e70d-c170-420f-8f8e-3436778817d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:28.239279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e70d-c170-420f-8f8e-3436778817d5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:28.239552 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e70d-c170-420f-8f8e-3436778817d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:28.239829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e70d-c170-420f-8f8e-3436778817d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:28.250595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955e70d-c170-420f-8f8e-3436778817d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:28.252296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955e70d-c170-420f-8f8e-3436778817d5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4955e70d-c170-420f-8f8e-3436778817d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:28.270849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:28.271923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:28.273103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:28.273103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:28.273103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:28.273103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:28.273103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 612/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:28.278324 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff65cd5-be05-4a0f-a1c1-bce998c4c30f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:28.387538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff65cd5-be05-4a0f-a1c1-bce998c4c30f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:28.390899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:28.431000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412a62f-89a1-4077-9317-c72d8d85edf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:28.431769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412a62f-89a1-4077-9317-c72d8d85edf9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:28.432136 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412a62f-89a1-4077-9317-c72d8d85edf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:28.432406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412a62f-89a1-4077-9317-c72d8d85edf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:28.433307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412a62f-89a1-4077-9317-c72d8d85edf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:28.433644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412a62f-89a1-4077-9317-c72d8d85edf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:28.435968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412a62f-89a1-4077-9317-c72d8d85edf9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:28.454688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 617/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:20:28.455955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955e70d-c170-420f-8f8e-3436778817d5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:28.466868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:28.570675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955e70d-c170-420f-8f8e-3436778817d5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:28.608328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de16713-94f4-458f-912e-c595dff44d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:28.609403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de16713-94f4-458f-912e-c595dff44d15 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:28.609873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de16713-94f4-458f-912e-c595dff44d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:28.610185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de16713-94f4-458f-912e-c595dff44d15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:28.617274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de16713-94f4-458f-912e-c595dff44d15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:28.618748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de16713-94f4-458f-912e-c595dff44d15 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7de16713-94f4-458f-912e-c595dff44d15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:20:28.633665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:28.635147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:28.636233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:20:28.636233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:20:28.636233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:20:28.636233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:20:28.636233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:20:28.636233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:20:28.636233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:28.636233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:28.636233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 613/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:20:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:28.641183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412a62f-89a1-4077-9317-c72d8d85edf9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:28.754943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412a62f-89a1-4077-9317-c72d8d85edf9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:28.790614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686d4a4-cb29-47a2-8e7d-65e8722444ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:28.791320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d4a4-cb29-47a2-8e7d-65e8722444ff None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:28.791693 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d4a4-cb29-47a2-8e7d-65e8722444ff None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:28.791975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d4a4-cb29-47a2-8e7d-65e8722444ff None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:28.792774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686d4a4-cb29-47a2-8e7d-65e8722444ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:28.793085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686d4a4-cb29-47a2-8e7d-65e8722444ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:28.796143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686d4a4-cb29-47a2-8e7d-65e8722444ff None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:28.810193 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 618/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:20:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:28.817983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de16713-94f4-458f-912e-c595dff44d15 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:28.893186 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:28.930793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de16713-94f4-458f-912e-c595dff44d15 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:28.968898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0af4e0-55ff-45db-b452-c9c19e016457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:28.969890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0af4e0-55ff-45db-b452-c9c19e016457 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:28.970250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0af4e0-55ff-45db-b452-c9c19e016457 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:28.970548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0af4e0-55ff-45db-b452-c9c19e016457 None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/1da2d85eb18049f196aeefc51d3f1e0a/roles/7541347b464141a78014fd57dccc765b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:28.987109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0af4e0-55ff-45db-b452-c9c19e016457 None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=1da2d85eb18049f196aeefc51d3f1e0a, role_id=7541347b464141a78014fd57dccc765b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:28.988551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0af4e0-55ff-45db-b452-c9c19e016457 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:29.001247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 614/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:20:28 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/1da2d85eb18049f196aeefc51d3f1e0a/roles/7541347b464141a78014fd57dccc765b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:20:31.049193 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3f0d0-abd5-4e63-9403-6ec64a158725 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:31.049717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3f0d0-abd5-4e63-9403-6ec64a158725 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:31.049819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3f0d0-abd5-4e63-9403-6ec64a158725 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:31.052893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 619/1233] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:20:31.056693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b4c5c-82fe-488f-8950-6749cc7d9903 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:31.057106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b4c5c-82fe-488f-8950-6749cc7d9903 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:31.057266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b4c5c-82fe-488f-8950-6749cc7d9903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:31.075019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5c6b4c5c-82fe-488f-8950-6749cc7d9903 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:20:31.076367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5c6b4c5c-82fe-488f-8950-6749cc7d9903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:20:31.147111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 615/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:20:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 06:20:31.171778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3f0d0-abd5-4e63-9403-6ec64a158725 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:31.197402 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:31.277128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3f0d0-abd5-4e63-9403-6ec64a158725 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:31.315325 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85840df8-1f21-41e9-a365-f324d93d1408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:31.316059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85840df8-1f21-41e9-a365-f324d93d1408 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:31.316341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85840df8-1f21-41e9-a365-f324d93d1408 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:31.316618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85840df8-1f21-41e9-a365-f324d93d1408 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:31.327563 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85840df8-1f21-41e9-a365-f324d93d1408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-85840df8-1f21-41e9-a365-f324d93d1408 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:20:31.335561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:31.336688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:31.337561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 620/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 06:20:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:31.343706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b4c5c-82fe-488f-8950-6749cc7d9903 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:31.395991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:31.456532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b4c5c-82fe-488f-8950-6749cc7d9903 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:31.491474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3638f-61e2-4084-8157-a384d2887d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:31.492148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3638f-61e2-4084-8157-a384d2887d80 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:31.492434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3638f-61e2-4084-8157-a384d2887d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:31.492718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3638f-61e2-4084-8157-a384d2887d80 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:31.493474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3638f-61e2-4084-8157-a384d2887d80 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:31.493784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3638f-61e2-4084-8157-a384d2887d80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:31.495203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3638f-61e2-4084-8157-a384d2887d80 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:31.504199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 616/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 06:20:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:20:31.510691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85840df8-1f21-41e9-a365-f324d93d1408 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:31.608873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:31.627482 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85840df8-1f21-41e9-a365-f324d93d1408 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:31.664874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc40f3-e0e8-4c8b-bf61-f210253709c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:31.665669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc40f3-e0e8-4c8b-bf61-f210253709c9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:31.665961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc40f3-e0e8-4c8b-bf61-f210253709c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:31.666255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc40f3-e0e8-4c8b-bf61-f210253709c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:31.677023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc40f3-e0e8-4c8b-bf61-f210253709c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:31.678501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc40f3-e0e8-4c8b-bf61-f210253709c9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f1fc40f3-e0e8-4c8b-bf61-f210253709c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:31.695643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:31.696756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:31.697662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:31.697662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:31.697662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:31.697662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:31.697662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 621/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:31.703352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3638f-61e2-4084-8157-a384d2887d80 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:31.809321 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3638f-61e2-4084-8157-a384d2887d80 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:31.821004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:31.849165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024045f-1ef2-405d-a0ab-b1ae91f3405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:31.849837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024045f-1ef2-405d-a0ab-b1ae91f3405b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:31.850164 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024045f-1ef2-405d-a0ab-b1ae91f3405b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:31.850464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024045f-1ef2-405d-a0ab-b1ae91f3405b None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:31.851187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024045f-1ef2-405d-a0ab-b1ae91f3405b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:31.851500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024045f-1ef2-405d-a0ab-b1ae91f3405b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:31.852933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024045f-1ef2-405d-a0ab-b1ae91f3405b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:31.865747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 617/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:31.871354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc40f3-e0e8-4c8b-bf61-f210253709c9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:31.980754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc40f3-e0e8-4c8b-bf61-f210253709c9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:32.016550 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd2b79-a847-4f72-b951-c0c89c2eee9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:32.017356 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd2b79-a847-4f72-b951-c0c89c2eee9c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:32.017657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd2b79-a847-4f72-b951-c0c89c2eee9c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:32.017917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd2b79-a847-4f72-b951-c0c89c2eee9c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:32.020896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:32.026677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd2b79-a847-4f72-b951-c0c89c2eee9c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:32.028264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd2b79-a847-4f72-b951-c0c89c2eee9c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-37fd2b79-a847-4f72-b951-c0c89c2eee9c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:20:32.043149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:32.044133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:32.046004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:32.046004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:32.046004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:20:32.046004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:20:32.046004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:20:32.046004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:20:32.046004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:20:32.046004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:20:32.046004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:32.046004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:32.049761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 622/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:32.053327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024045f-1ef2-405d-a0ab-b1ae91f3405b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:32.153052 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024045f-1ef2-405d-a0ab-b1ae91f3405b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:32.191591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7982c-f1b1-4394-82dd-9c4e00aeea1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:32.192274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7982c-f1b1-4394-82dd-9c4e00aeea1d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:32.192558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7982c-f1b1-4394-82dd-9c4e00aeea1d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:32.192838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7982c-f1b1-4394-82dd-9c4e00aeea1d None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:32.193694 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7982c-f1b1-4394-82dd-9c4e00aeea1d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:32.194030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7982c-f1b1-4394-82dd-9c4e00aeea1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:32.195459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7982c-f1b1-4394-82dd-9c4e00aeea1d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:32.218895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 618/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 06:20:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:32.224751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd2b79-a847-4f72-b951-c0c89c2eee9c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:32.338870 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd2b79-a847-4f72-b951-c0c89c2eee9c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:32.373665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77be15-03f5-4ce5-89f7-7dc7513b1a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:32.374356 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77be15-03f5-4ce5-89f7-7dc7513b1a87 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:32.374647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77be15-03f5-4ce5-89f7-7dc7513b1a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:32.374969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77be15-03f5-4ce5-89f7-7dc7513b1a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:32.449193 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:32.458455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77be15-03f5-4ce5-89f7-7dc7513b1a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:32.459992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77be15-03f5-4ce5-89f7-7dc7513b1a87 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:32.570111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 623/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:20:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 06:20:32.571392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7982c-f1b1-4394-82dd-9c4e00aeea1d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:32.599509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:32.674844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7982c-f1b1-4394-82dd-9c4e00aeea1d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:32.722629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca5b01-8758-4799-bbdb-96bd6c2b2fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:32.723431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca5b01-8758-4799-bbdb-96bd6c2b2fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:32.723713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca5b01-8758-4799-bbdb-96bd6c2b2fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:32.723992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca5b01-8758-4799-bbdb-96bd6c2b2fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:32.734659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca5b01-8758-4799-bbdb-96bd6c2b2fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:32.736062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca5b01-8758-4799-bbdb-96bd6c2b2fc8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-58ca5b01-8758-4799-bbdb-96bd6c2b2fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:32.757420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:32.758528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:32.759419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:32.759419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:32.759419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:32.759419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:32.759419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 619/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:32.765197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77be15-03f5-4ce5-89f7-7dc7513b1a87 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:32.876070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:32.883007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77be15-03f5-4ce5-89f7-7dc7513b1a87 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:32.937274 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3595576-cea8-4ba3-91f5-3f7b5d7851cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:32.937963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3595576-cea8-4ba3-91f5-3f7b5d7851cc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:32.938261 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3595576-cea8-4ba3-91f5-3f7b5d7851cc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:32.938536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3595576-cea8-4ba3-91f5-3f7b5d7851cc None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:32.939270 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3595576-cea8-4ba3-91f5-3f7b5d7851cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:32.939648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3595576-cea8-4ba3-91f5-3f7b5d7851cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:32.941273 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3595576-cea8-4ba3-91f5-3f7b5d7851cc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:32.959066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 624/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:32.964474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca5b01-8758-4799-bbdb-96bd6c2b2fc8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:33.051926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:33.078706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca5b01-8758-4799-bbdb-96bd6c2b2fc8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:33.120619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09666e0-6475-4ea7-a2cd-b4ac6867b419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:33.121426 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09666e0-6475-4ea7-a2cd-b4ac6867b419 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:33.121713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09666e0-6475-4ea7-a2cd-b4ac6867b419 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:33.122028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09666e0-6475-4ea7-a2cd-b4ac6867b419 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:33.130372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09666e0-6475-4ea7-a2cd-b4ac6867b419 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:33.131991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09666e0-6475-4ea7-a2cd-b4ac6867b419 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f09666e0-6475-4ea7-a2cd-b4ac6867b419 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:20:33.150316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:33.151747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:33.152635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:20:33.152635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:20:33.152635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:20:33.152635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:20:33.152635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:20:33.152635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:20:33.152635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:33.152635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:33.152635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 620/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:20:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 06:20:33.159078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3595576-cea8-4ba3-91f5-3f7b5d7851cc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:33.274132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3595576-cea8-4ba3-91f5-3f7b5d7851cc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:33.314116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa02d4b-8963-4d09-874e-e8c1c26af8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:33.314718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa02d4b-8963-4d09-874e-e8c1c26af8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:33.315005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa02d4b-8963-4d09-874e-e8c1c26af8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:33.315283 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa02d4b-8963-4d09-874e-e8c1c26af8a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:33.316089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa02d4b-8963-4d09-874e-e8c1c26af8a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:33.316390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa02d4b-8963-4d09-874e-e8c1c26af8a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:33.317843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa02d4b-8963-4d09-874e-e8c1c26af8a4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:33.331705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 625/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:20:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:33.337789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09666e0-6475-4ea7-a2cd-b4ac6867b419 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:33.444309 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09666e0-6475-4ea7-a2cd-b4ac6867b419 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:33.480461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ed72c-4631-4d8f-8000-d0de3cdcc10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:33.481134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed72c-4631-4d8f-8000-d0de3cdcc10a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:33.481419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed72c-4631-4d8f-8000-d0de3cdcc10a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:33.482194 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed72c-4631-4d8f-8000-d0de3cdcc10a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:33.482916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ed72c-4631-4d8f-8000-d0de3cdcc10a None admin] RBAC: Adding query filter params (role.id=7541347b464141a78014fd57dccc765b, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=1da2d85eb18049f196aeefc51d3f1e0a) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:33.483225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ed72c-4631-4d8f-8000-d0de3cdcc10a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:33.484715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ed72c-4631-4d8f-8000-d0de3cdcc10a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:33.491378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 621/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:20:33 2022] GET /identity/v3/role_assignments?user.id=1da2d85eb18049f196aeefc51d3f1e0a&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=7541347b464141a78014fd57dccc765b => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:35.675798 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7c7df-72c9-47cf-87ca-1da5b2709b41 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:35.676347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7c7df-72c9-47cf-87ca-1da5b2709b41 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:35.676347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7c7df-72c9-47cf-87ca-1da5b2709b41 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:35.677721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 626/1247] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:20:35.683283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6607e8-d4cc-43d5-b2b3-eb47f143a5d9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:35.683721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6607e8-d4cc-43d5-b2b3-eb47f143a5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:35.683881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6607e8-d4cc-43d5-b2b3-eb47f143a5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:35.690178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:35.704308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6d6607e8-d4cc-43d5-b2b3-eb47f143a5d9 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:20:35.705505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6d6607e8-d4cc-43d5-b2b3-eb47f143a5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:20:35.780461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 622/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:20:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:20:35.806558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7c7df-72c9-47cf-87ca-1da5b2709b41 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:35.829538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:35.938803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7c7df-72c9-47cf-87ca-1da5b2709b41 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:35.987857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17b165-4934-4081-bcb1-76698f6ae1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:35.988597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17b165-4934-4081-bcb1-76698f6ae1fe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:35.988883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17b165-4934-4081-bcb1-76698f6ae1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:35.989165 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17b165-4934-4081-bcb1-76698f6ae1fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:36.000467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17b165-4934-4081-bcb1-76698f6ae1fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4c17b165-4934-4081-bcb1-76698f6ae1fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:20:36.008191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:36.009191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:36.010066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 627/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:20:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:36.017676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6607e8-d4cc-43d5-b2b3-eb47f143a5d9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:36.136082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6607e8-d4cc-43d5-b2b3-eb47f143a5d9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:36.170744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:36.176728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ea70f-ffa1-4b10-9a2e-4c8de0b2f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:36.177482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ea70f-ffa1-4b10-9a2e-4c8de0b2f85f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:36.177831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ea70f-ffa1-4b10-9a2e-4c8de0b2f85f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:36.178234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ea70f-ffa1-4b10-9a2e-4c8de0b2f85f None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:36.179217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ea70f-ffa1-4b10-9a2e-4c8de0b2f85f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:36.179793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ea70f-ffa1-4b10-9a2e-4c8de0b2f85f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:36.181605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ea70f-ffa1-4b10-9a2e-4c8de0b2f85f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:36.190736 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 623/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:20:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:36.197272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17b165-4934-4081-bcb1-76698f6ae1fe None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:36.302381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:36.331511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17b165-4934-4081-bcb1-76698f6ae1fe None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:36.367829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ca442-4b28-4693-a541-5080c99e6378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:36.368634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ca442-4b28-4693-a541-5080c99e6378 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:36.368920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ca442-4b28-4693-a541-5080c99e6378 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:36.369204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ca442-4b28-4693-a541-5080c99e6378 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:36.380405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ca442-4b28-4693-a541-5080c99e6378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:36.381875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ca442-4b28-4693-a541-5080c99e6378 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-179ca442-4b28-4693-a541-5080c99e6378 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:36.399510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:36.400507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:36.401707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:36.401707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:36.401707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:36.401707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:36.401707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 628/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:36.409606 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ea70f-ffa1-4b10-9a2e-4c8de0b2f85f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:36.532039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ea70f-ffa1-4b10-9a2e-4c8de0b2f85f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:36.566190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28c48d-1b78-4ebd-af9b-37508bfc4cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:36.566845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28c48d-1b78-4ebd-af9b-37508bfc4cda None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:36.567133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28c48d-1b78-4ebd-af9b-37508bfc4cda None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:36.567412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28c48d-1b78-4ebd-af9b-37508bfc4cda None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:36.568140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28c48d-1b78-4ebd-af9b-37508bfc4cda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:36.568452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28c48d-1b78-4ebd-af9b-37508bfc4cda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:36.569855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28c48d-1b78-4ebd-af9b-37508bfc4cda None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:36.584798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 624/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:36.590829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ca442-4b28-4693-a541-5080c99e6378 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:36.711884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ca442-4b28-4693-a541-5080c99e6378 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:36.742036 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:36.751067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51048-109f-4e8b-aafe-e5b9aea40ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:36.751879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51048-109f-4e8b-aafe-e5b9aea40ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:36.752162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51048-109f-4e8b-aafe-e5b9aea40ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:36.752460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51048-109f-4e8b-aafe-e5b9aea40ca9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:36.760423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd51048-109f-4e8b-aafe-e5b9aea40ca9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:36.761874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd51048-109f-4e8b-aafe-e5b9aea40ca9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0cd51048-109f-4e8b-aafe-e5b9aea40ca9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:20:36.776479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:36.777671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:36.778670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:36.778670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:36.778670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:20:36.778670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:20:36.778670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:20:36.778670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:20:36.778670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:20:36.778670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:20:36.778670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:36.778670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:36.778670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 629/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:36.784009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28c48d-1b78-4ebd-af9b-37508bfc4cda None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:36.844862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:36.906505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28c48d-1b78-4ebd-af9b-37508bfc4cda None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:36.947968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32113fc-2a97-4b8b-a6cf-9363d56af3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:36.948738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32113fc-2a97-4b8b-a6cf-9363d56af3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:36.949117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32113fc-2a97-4b8b-a6cf-9363d56af3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:36.949481 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32113fc-2a97-4b8b-a6cf-9363d56af3a0 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:36.950586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32113fc-2a97-4b8b-a6cf-9363d56af3a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:36.950917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32113fc-2a97-4b8b-a6cf-9363d56af3a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:36.952604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32113fc-2a97-4b8b-a6cf-9363d56af3a0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:36.972159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 625/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 06:20:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:20:36.979243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51048-109f-4e8b-aafe-e5b9aea40ca9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:37.087840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51048-109f-4e8b-aafe-e5b9aea40ca9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:37.129543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73903fde-2512-4bf3-ad7c-9db6f865e7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:37.130260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73903fde-2512-4bf3-ad7c-9db6f865e7ef None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:37.130566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73903fde-2512-4bf3-ad7c-9db6f865e7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:37.130841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73903fde-2512-4bf3-ad7c-9db6f865e7ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:37.192008 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:37.206752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73903fde-2512-4bf3-ad7c-9db6f865e7ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:37.208231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73903fde-2512-4bf3-ad7c-9db6f865e7ef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:37.322654 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 630/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:20:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:20:37.329586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32113fc-2a97-4b8b-a6cf-9363d56af3a0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:37.441338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32113fc-2a97-4b8b-a6cf-9363d56af3a0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:37.456338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:37.479195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aba4d3-e5b8-4934-87b2-b5fc5cb91b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:37.480094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aba4d3-e5b8-4934-87b2-b5fc5cb91b2b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:37.480384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aba4d3-e5b8-4934-87b2-b5fc5cb91b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:37.480665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aba4d3-e5b8-4934-87b2-b5fc5cb91b2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:37.493384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aba4d3-e5b8-4934-87b2-b5fc5cb91b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:37.495401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aba4d3-e5b8-4934-87b2-b5fc5cb91b2b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-60aba4d3-e5b8-4934-87b2-b5fc5cb91b2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:37.517965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:37.519068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:37.520038 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:37.520038 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:37.520038 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:37.520038 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:37.520038 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 626/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:37.525369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73903fde-2512-4bf3-ad7c-9db6f865e7ef None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:37.633889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:37.650910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73903fde-2512-4bf3-ad7c-9db6f865e7ef None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:37.691506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfdaf4-d6ba-438c-9c1a-88959122b373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:37.692193 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfdaf4-d6ba-438c-9c1a-88959122b373 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:37.692484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfdaf4-d6ba-438c-9c1a-88959122b373 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:37.692746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfdaf4-d6ba-438c-9c1a-88959122b373 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:37.693507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfdaf4-d6ba-438c-9c1a-88959122b373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:37.693819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfdaf4-d6ba-438c-9c1a-88959122b373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:37.695227 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfdaf4-d6ba-438c-9c1a-88959122b373 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:37.707482 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 631/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:20:37.713196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aba4d3-e5b8-4934-87b2-b5fc5cb91b2b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:37.819995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aba4d3-e5b8-4934-87b2-b5fc5cb91b2b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:37.862275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e3b03-ef14-44f5-b0ef-84881f72a4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:37.863002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3b03-ef14-44f5-b0ef-84881f72a4bc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:37.863326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3b03-ef14-44f5-b0ef-84881f72a4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:37.863606 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3b03-ef14-44f5-b0ef-84881f72a4bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:37.870870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e3b03-ef14-44f5-b0ef-84881f72a4bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:37.872382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e3b03-ef14-44f5-b0ef-84881f72a4bc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-806e3b03-ef14-44f5-b0ef-84881f72a4bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:20:37.886762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:37.888396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:37.889447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:20:37.889447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:20:37.889447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:20:37.889447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:20:37.889447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:20:37.889447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:20:37.889447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:37.889447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:37.889447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 627/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:20:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:37.894252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfdaf4-d6ba-438c-9c1a-88959122b373 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:38.006716 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfdaf4-d6ba-438c-9c1a-88959122b373 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:38.045845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018102ad-992a-4c7b-aec7-c736bfaf8993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:38.046558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018102ad-992a-4c7b-aec7-c736bfaf8993 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:38.046840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018102ad-992a-4c7b-aec7-c736bfaf8993 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:38.047115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018102ad-992a-4c7b-aec7-c736bfaf8993 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:38.047915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018102ad-992a-4c7b-aec7-c736bfaf8993 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:38.048225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018102ad-992a-4c7b-aec7-c736bfaf8993 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:38.049678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018102ad-992a-4c7b-aec7-c736bfaf8993 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:38.065825 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 632/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:20:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:20:38.072051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e3b03-ef14-44f5-b0ef-84881f72a4bc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:38.099945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:38.179936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e3b03-ef14-44f5-b0ef-84881f72a4bc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:38.216143 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4a9eb-93b9-4ca4-93dc-53f5cbe1618c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:38.216999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4a9eb-93b9-4ca4-93dc-53f5cbe1618c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:38.217285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4a9eb-93b9-4ca4-93dc-53f5cbe1618c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:38.217566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4a9eb-93b9-4ca4-93dc-53f5cbe1618c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:38.218462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4a9eb-93b9-4ca4-93dc-53f5cbe1618c None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=1da2d85eb18049f196aeefc51d3f1e0a) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:38.218778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4a9eb-93b9-4ca4-93dc-53f5cbe1618c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:38.220253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4a9eb-93b9-4ca4-93dc-53f5cbe1618c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:38.226782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 628/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:20:38 2022] GET /identity/v3/role_assignments?user.id=1da2d85eb18049f196aeefc51d3f1e0a&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:40.230392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b86ab-50d9-40a1-b064-b673832cdd44 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:40.231736 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b86ab-50d9-40a1-b064-b673832cdd44 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:40.232031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b86ab-50d9-40a1-b064-b673832cdd44 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:40.233396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 633/1261] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:20:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:20:40.238589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd5640-8896-402e-817b-e8c4a8e5a1e4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:40.238992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd5640-8896-402e-817b-e8c4a8e5a1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:40.239147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd5640-8896-402e-817b-e8c4a8e5a1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:40.255338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e2fd5640-8896-402e-817b-e8c4a8e5a1e4 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:20:40.256497 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e2fd5640-8896-402e-817b-e8c4a8e5a1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:20:40.322060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 629/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:20:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:20:40.344507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b86ab-50d9-40a1-b064-b673832cdd44 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:40.355217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:40.445472 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b86ab-50d9-40a1-b064-b673832cdd44 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:40.479677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9784c-bb34-40f0-9464-13cc19607ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:40.480451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9784c-bb34-40f0-9464-13cc19607ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:40.481689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9784c-bb34-40f0-9464-13cc19607ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:40.481994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9784c-bb34-40f0-9464-13cc19607ca6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:40.491753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9784c-bb34-40f0-9464-13cc19607ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8ba9784c-bb34-40f0-9464-13cc19607ca6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:20:40.500236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:40.501320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:40.502381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 634/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:20:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:40.507333 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd5640-8896-402e-817b-e8c4a8e5a1e4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:40.526098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:40.604552 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd5640-8896-402e-817b-e8c4a8e5a1e4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:40.637709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce089488-b894-4f63-9db6-ce5d879d631f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:40.638384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce089488-b894-4f63-9db6-ce5d879d631f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:40.638670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce089488-b894-4f63-9db6-ce5d879d631f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:40.638962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce089488-b894-4f63-9db6-ce5d879d631f None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:40.639712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce089488-b894-4f63-9db6-ce5d879d631f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:40.640018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce089488-b894-4f63-9db6-ce5d879d631f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:40.641368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce089488-b894-4f63-9db6-ce5d879d631f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:40.649056 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 630/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:20:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:40.655682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9784c-bb34-40f0-9464-13cc19607ca6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:40.721042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:40.759357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9784c-bb34-40f0-9464-13cc19607ca6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:40.793001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e750b-db19-4c9b-87ef-ac726e5bf4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:40.793978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e750b-db19-4c9b-87ef-ac726e5bf4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:40.794267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e750b-db19-4c9b-87ef-ac726e5bf4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:40.794548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e750b-db19-4c9b-87ef-ac726e5bf4b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:40.809995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e750b-db19-4c9b-87ef-ac726e5bf4b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:40.811584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e750b-db19-4c9b-87ef-ac726e5bf4b7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-474e750b-db19-4c9b-87ef-ac726e5bf4b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:40.830405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:40.831420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:40.832350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:40.832350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:40.832350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:40.832350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:40.832350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 635/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:40.838085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce089488-b894-4f63-9db6-ce5d879d631f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:40.923856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:40.943094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce089488-b894-4f63-9db6-ce5d879d631f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:40.975506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58450f07-1ec1-4154-b2b8-3f1f71430af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:40.976172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58450f07-1ec1-4154-b2b8-3f1f71430af6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:40.976459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58450f07-1ec1-4154-b2b8-3f1f71430af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:40.976741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58450f07-1ec1-4154-b2b8-3f1f71430af6 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:40.977469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58450f07-1ec1-4154-b2b8-3f1f71430af6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:40.977782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58450f07-1ec1-4154-b2b8-3f1f71430af6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:40.979177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58450f07-1ec1-4154-b2b8-3f1f71430af6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:40.990729 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 631/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:20:40.997078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e750b-db19-4c9b-87ef-ac726e5bf4b7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:41.096047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e750b-db19-4c9b-87ef-ac726e5bf4b7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:41.114954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:41.129290 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50766e-22ea-4525-adee-7e38c6d534fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:41.130117 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50766e-22ea-4525-adee-7e38c6d534fc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:41.130405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50766e-22ea-4525-adee-7e38c6d534fc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:41.130680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50766e-22ea-4525-adee-7e38c6d534fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:41.138942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50766e-22ea-4525-adee-7e38c6d534fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:41.140381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50766e-22ea-4525-adee-7e38c6d534fc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bb50766e-22ea-4525-adee-7e38c6d534fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:20:41.154818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:41.155987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:41.157340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:41.157340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:41.157340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:20:41.157340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:20:41.157340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:20:41.157340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:20:41.157340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:20:41.157340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:20:41.157340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:41.157340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:41.157340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 636/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:41.161785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58450f07-1ec1-4154-b2b8-3f1f71430af6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:41.257086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58450f07-1ec1-4154-b2b8-3f1f71430af6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:41.289000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6977cc-ab22-4731-958c-07d841f0e0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:41.289651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6977cc-ab22-4731-958c-07d841f0e0df None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:41.289931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6977cc-ab22-4731-958c-07d841f0e0df None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:41.290251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6977cc-ab22-4731-958c-07d841f0e0df None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:41.291135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6977cc-ab22-4731-958c-07d841f0e0df None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:41.291443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6977cc-ab22-4731-958c-07d841f0e0df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:41.292858 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6977cc-ab22-4731-958c-07d841f0e0df None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:41.302252 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:41.311079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 632/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 06:20:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:41.316713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50766e-22ea-4525-adee-7e38c6d534fc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:41.415321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50766e-22ea-4525-adee-7e38c6d534fc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:41.448653 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52c613-cd3a-4600-b366-6e7297d82daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:41.449868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52c613-cd3a-4600-b366-6e7297d82daf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:41.450178 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52c613-cd3a-4600-b366-6e7297d82daf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:41.450456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52c613-cd3a-4600-b366-6e7297d82daf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:41.460548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52c613-cd3a-4600-b366-6e7297d82daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:41.462002 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52c613-cd3a-4600-b366-6e7297d82daf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7b52c613-cd3a-4600-b366-6e7297d82daf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:41.478863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:41.479817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:41.481229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:41.481229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:41.481229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:41.481229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:41.481229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 637/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:41.486119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6977cc-ab22-4731-958c-07d841f0e0df None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:41.583836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6977cc-ab22-4731-958c-07d841f0e0df None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:41.617138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f03fb-5e7c-4538-b37a-699463251a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:41.617792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f03fb-5e7c-4538-b37a-699463251a71 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:41.618111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f03fb-5e7c-4538-b37a-699463251a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:41.618390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f03fb-5e7c-4538-b37a-699463251a71 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:41.619110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f03fb-5e7c-4538-b37a-699463251a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:41.619417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f03fb-5e7c-4538-b37a-699463251a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:41.620771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f03fb-5e7c-4538-b37a-699463251a71 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:41.631553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 633/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:41.637381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52c613-cd3a-4600-b366-6e7297d82daf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:41.670443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:41.738182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52c613-cd3a-4600-b366-6e7297d82daf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:41.770413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792fc8e-3037-4364-b212-cd4f64890c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:41.771199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792fc8e-3037-4364-b212-cd4f64890c7c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:41.771485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792fc8e-3037-4364-b212-cd4f64890c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:41.771765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792fc8e-3037-4364-b212-cd4f64890c7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:41.779137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792fc8e-3037-4364-b212-cd4f64890c7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:41.780575 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792fc8e-3037-4364-b212-cd4f64890c7c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3792fc8e-3037-4364-b212-cd4f64890c7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:20:41.795723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:41.796754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:41.797876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:20:41.797876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:20:41.797876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:20:41.797876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:20:41.797876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:20:41.797876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:20:41.797876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:41.797876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:41.797876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 638/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:20:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:41.802835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f03fb-5e7c-4538-b37a-699463251a71 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:41.867433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:41.908461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f03fb-5e7c-4538-b37a-699463251a71 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:41.940392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108226c8-acf7-4bf8-bbdd-59659739669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:41.941041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108226c8-acf7-4bf8-bbdd-59659739669c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:41.941324 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108226c8-acf7-4bf8-bbdd-59659739669c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:41.941600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108226c8-acf7-4bf8-bbdd-59659739669c None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:41.942407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108226c8-acf7-4bf8-bbdd-59659739669c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:41.942714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108226c8-acf7-4bf8-bbdd-59659739669c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:41.944151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108226c8-acf7-4bf8-bbdd-59659739669c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:41.956080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 634/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:20:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:41.962063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792fc8e-3037-4364-b212-cd4f64890c7c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:42.052226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:42.061992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792fc8e-3037-4364-b212-cd4f64890c7c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:42.096820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26740be-38c6-497b-9dc7-42c2b88b9671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:42.097695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26740be-38c6-497b-9dc7-42c2b88b9671 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:42.098001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26740be-38c6-497b-9dc7-42c2b88b9671 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:42.098296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26740be-38c6-497b-9dc7-42c2b88b9671 None admin] PATH_INFO: `/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/1da2d85eb18049f196aeefc51d3f1e0a/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:42.100209 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26740be-38c6-497b-9dc7-42c2b88b9671 None admin] RBAC: Authorizing `identity:create_grant(project_id=91ccc1b22a434cf8a9be253de3ee3862, user_id=1da2d85eb18049f196aeefc51d3f1e0a, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:42.101646 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26740be-38c6-497b-9dc7-42c2b88b9671 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:42.114727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 639/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 06:20:41 2022] PUT /identity/v3/projects/91ccc1b22a434cf8a9be253de3ee3862/users/1da2d85eb18049f196aeefc51d3f1e0a/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:20:44.127684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243519fa-66ee-4a28-ab9e-80051235b8ce None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:44.128230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243519fa-66ee-4a28-ab9e-80051235b8ce None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:44.128230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243519fa-66ee-4a28-ab9e-80051235b8ce None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:44.129620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 635/1274] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:20:44.134860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fd1d0-0e64-4a8c-9197-d8e71d315da7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:44.135248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fd1d0-0e64-4a8c-9197-d8e71d315da7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:44.135420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fd1d0-0e64-4a8c-9197-d8e71d315da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:44.153050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4e4fd1d0-0e64-4a8c-9197-d8e71d315da7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:20:44.154249 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4e4fd1d0-0e64-4a8c-9197-d8e71d315da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:20:44.219285 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 640/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:20:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:20:44.242974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243519fa-66ee-4a28-ab9e-80051235b8ce None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:44.342253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243519fa-66ee-4a28-ab9e-80051235b8ce None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:44.359528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:44.376708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8957bbe-74c9-4b52-9ad1-0fd5525a8517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:44.377500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8957bbe-74c9-4b52-9ad1-0fd5525a8517 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:44.377786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8957bbe-74c9-4b52-9ad1-0fd5525a8517 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:44.378104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8957bbe-74c9-4b52-9ad1-0fd5525a8517 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:44.387996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8957bbe-74c9-4b52-9ad1-0fd5525a8517 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d8957bbe-74c9-4b52-9ad1-0fd5525a8517 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 06:20:44.395642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:44.396769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:44.398043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 636/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 06:20:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:44.403128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fd1d0-0e64-4a8c-9197-d8e71d315da7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:44.481879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:44.500553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fd1d0-0e64-4a8c-9197-d8e71d315da7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:44.533707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3c453-f6bd-46bc-a558-16b4fdac6311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:44.534384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3c453-f6bd-46bc-a558-16b4fdac6311 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:44.534669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3c453-f6bd-46bc-a558-16b4fdac6311 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:44.534944 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3c453-f6bd-46bc-a558-16b4fdac6311 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:44.535682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3c453-f6bd-46bc-a558-16b4fdac6311 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:44.535988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3c453-f6bd-46bc-a558-16b4fdac6311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:44.537348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3c453-f6bd-46bc-a558-16b4fdac6311 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:44.545728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 641/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 06:20:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:44.551298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8957bbe-74c9-4b52-9ad1-0fd5525a8517 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:44.647226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8957bbe-74c9-4b52-9ad1-0fd5525a8517 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:44.679785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfe62a-9479-4524-a6bf-471825522052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:44.680552 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe62a-9479-4524-a6bf-471825522052 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:44.680834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe62a-9479-4524-a6bf-471825522052 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:44.681112 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe62a-9479-4524-a6bf-471825522052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:44.691194 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfe62a-9479-4524-a6bf-471825522052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:44.692614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfe62a-9479-4524-a6bf-471825522052 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7fbfe62a-9479-4524-a6bf-471825522052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:44.708933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:44.710036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:44.715588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:44.715588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:44.715588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:44.715588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:44.715588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 637/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:44.716289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3c453-f6bd-46bc-a558-16b4fdac6311 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:44.817508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3c453-f6bd-46bc-a558-16b4fdac6311 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:44.853057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a941b-0f36-40fc-971a-f19e602a9fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:44.853717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a941b-0f36-40fc-971a-f19e602a9fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:44.854029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a941b-0f36-40fc-971a-f19e602a9fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:44.854307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a941b-0f36-40fc-971a-f19e602a9fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:44.855025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a941b-0f36-40fc-971a-f19e602a9fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:44.855447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a941b-0f36-40fc-971a-f19e602a9fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:44.856899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a941b-0f36-40fc-971a-f19e602a9fc5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:44.868437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 642/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:20:44.874639 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfe62a-9479-4524-a6bf-471825522052 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:44.905505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:44.973250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfe62a-9479-4524-a6bf-471825522052 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:45.006515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05b2ec-99db-4494-905a-f53031689d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:45.007308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05b2ec-99db-4494-905a-f53031689d36 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:45.007594 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05b2ec-99db-4494-905a-f53031689d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:45.007873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05b2ec-99db-4494-905a-f53031689d36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:45.015262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05b2ec-99db-4494-905a-f53031689d36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:45.016689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05b2ec-99db-4494-905a-f53031689d36 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-eb05b2ec-99db-4494-905a-f53031689d36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 06:20:45.031180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:45.032283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:45.033627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:45.033627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:45.033627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 06:20:45.033627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 06:20:45.033627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 06:20:45.033627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 06:20:45.033627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 06:20:45.033627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 06:20:45.033627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 06:20:45.033627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:45.033627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 638/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:45.038181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a941b-0f36-40fc-971a-f19e602a9fc5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:45.041931 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:45.136396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a941b-0f36-40fc-971a-f19e602a9fc5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:45.169863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c938b7-16c5-4eb6-95ff-cc545b17f467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:45.170491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c938b7-16c5-4eb6-95ff-cc545b17f467 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:45.170779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c938b7-16c5-4eb6-95ff-cc545b17f467 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:45.171056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c938b7-16c5-4eb6-95ff-cc545b17f467 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:45.171904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c938b7-16c5-4eb6-95ff-cc545b17f467 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:45.172216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c938b7-16c5-4eb6-95ff-cc545b17f467 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:45.173640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c938b7-16c5-4eb6-95ff-cc545b17f467 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:45.190924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 643/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 06:20:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:45.196569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05b2ec-99db-4494-905a-f53031689d36 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:45.284313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:45.294703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05b2ec-99db-4494-905a-f53031689d36 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:45.327413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f6810-669e-45de-abff-18a0b4ac04e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:45.328051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f6810-669e-45de-abff-18a0b4ac04e7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:45.328336 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f6810-669e-45de-abff-18a0b4ac04e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:45.328614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f6810-669e-45de-abff-18a0b4ac04e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:45.398254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f6810-669e-45de-abff-18a0b4ac04e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:45.399957 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f6810-669e-45de-abff-18a0b4ac04e7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:45.486220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:45.496395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 639/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 06:20:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 06:20:45.502139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c938b7-16c5-4eb6-95ff-cc545b17f467 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:45.563134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:45.602296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c938b7-16c5-4eb6-95ff-cc545b17f467 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:45.637179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01337cc-2d41-4b4e-a406-90674babe2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:45.637974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01337cc-2d41-4b4e-a406-90674babe2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:45.638256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01337cc-2d41-4b4e-a406-90674babe2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:45.638536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01337cc-2d41-4b4e-a406-90674babe2e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:45.648173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01337cc-2d41-4b4e-a406-90674babe2e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:45.649876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01337cc-2d41-4b4e-a406-90674babe2e9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b01337cc-2d41-4b4e-a406-90674babe2e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 06:20:45.669085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:45.670170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:45.671173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 06:20:45.671173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 06:20:45.671173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 06:20:45.671173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:45.671173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 644/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:45.675971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f6810-669e-45de-abff-18a0b4ac04e7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:45.770485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f6810-669e-45de-abff-18a0b4ac04e7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:45.802989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8d201-cc4d-4f0e-bdec-2e59aee2601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:45.803628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8d201-cc4d-4f0e-bdec-2e59aee2601e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:45.803910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8d201-cc4d-4f0e-bdec-2e59aee2601e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:45.804193 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8d201-cc4d-4f0e-bdec-2e59aee2601e None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:45.804903 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8d201-cc4d-4f0e-bdec-2e59aee2601e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:45.805208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8d201-cc4d-4f0e-bdec-2e59aee2601e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:45.806571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8d201-cc4d-4f0e-bdec-2e59aee2601e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:45.819723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 640/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 06:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:45.827031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01337cc-2d41-4b4e-a406-90674babe2e9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:45.924363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01337cc-2d41-4b4e-a406-90674babe2e9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:45.937682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:45.957867 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de7ab8-90a6-496c-955b-1dd77e5e3d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:45.958667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de7ab8-90a6-496c-955b-1dd77e5e3d50 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:45.958947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de7ab8-90a6-496c-955b-1dd77e5e3d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:45.959220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de7ab8-90a6-496c-955b-1dd77e5e3d50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:45.966315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de7ab8-90a6-496c-955b-1dd77e5e3d50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:45.967742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de7ab8-90a6-496c-955b-1dd77e5e3d50 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-06de7ab8-90a6-496c-955b-1dd77e5e3d50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 06:20:45.982819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:45.983876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:45.985194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 06:20:45.985194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 06:20:45.985194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 06:20:45.985194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 06:20:45.985194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 06:20:45.985194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 06:20:45.985194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 06:20:45.985194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 21 06:20:45.985194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 645/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 06:20:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:45.990586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8d201-cc4d-4f0e-bdec-2e59aee2601e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:46.039664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:46.089135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8d201-cc4d-4f0e-bdec-2e59aee2601e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:46.121075 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b2285-dd3a-428b-b407-e8cb3ba1e779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:46.121729 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2285-dd3a-428b-b407-e8cb3ba1e779 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:46.122042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2285-dd3a-428b-b407-e8cb3ba1e779 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:46.122319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2285-dd3a-428b-b407-e8cb3ba1e779 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:46.123095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b2285-dd3a-428b-b407-e8cb3ba1e779 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:46.123403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b2285-dd3a-428b-b407-e8cb3ba1e779 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:46.124812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b2285-dd3a-428b-b407-e8cb3ba1e779 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:46.136849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 641/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 06:20:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:46.142938 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de7ab8-90a6-496c-955b-1dd77e5e3d50 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:46.239878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de7ab8-90a6-496c-955b-1dd77e5e3d50 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:46.272936 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed94a2-fc49-4650-875f-d667cbf3356e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:46.273570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed94a2-fc49-4650-875f-d667cbf3356e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:46.273854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed94a2-fc49-4650-875f-d667cbf3356e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:46.274160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed94a2-fc49-4650-875f-d667cbf3356e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:46.275245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed94a2-fc49-4650-875f-d667cbf3356e None admin] RBAC: Adding query filter params (role.id=737dda9a674348dc8a8afe8f7827fa89, scope.project.id=91ccc1b22a434cf8a9be253de3ee3862, user.id=1da2d85eb18049f196aeefc51d3f1e0a) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:46.275555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed94a2-fc49-4650-875f-d667cbf3356e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:46.277012 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed94a2-fc49-4650-875f-d667cbf3356e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:46.282773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 646/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 06:20:46 2022] GET /identity/v3/role_assignments?user.id=1da2d85eb18049f196aeefc51d3f1e0a&scope.project.id=91ccc1b22a434cf8a9be253de3ee3862&role.id=737dda9a674348dc8a8afe8f7827fa89 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:48.320222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f731-c6be-4c6f-9e27-9b537ca23fd4 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:48.320750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f731-c6be-4c6f-9e27-9b537ca23fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:48.320750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f731-c6be-4c6f-9e27-9b537ca23fd4 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:48.322125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 642/1288] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:20:48.327373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d8a1-e060-4734-a5d6-3742921c7ab1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:48.327817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d8a1-e060-4734-a5d6-3742921c7ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:48.327961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d8a1-e060-4734-a5d6-3742921c7ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:48.349689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e646d8a1-e060-4734-a5d6-3742921c7ab1 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:20:48.350894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e646d8a1-e060-4734-a5d6-3742921c7ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:20:48.379840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:48.418133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 647/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:20:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:20:48.439891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977f731-c6be-4c6f-9e27-9b537ca23fd4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:48.538967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977f731-c6be-4c6f-9e27-9b537ca23fd4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:48.550193 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:48.573082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f4338-c52a-44e7-8ed9-0653e4185677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:48.574287 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f4338-c52a-44e7-8ed9-0653e4185677 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:48.574572 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f4338-c52a-44e7-8ed9-0653e4185677 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:48.574848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f4338-c52a-44e7-8ed9-0653e4185677 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:48.575492 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f4338-c52a-44e7-8ed9-0653e4185677 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:48.576836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f4338-c52a-44e7-8ed9-0653e4185677 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-395f4338-c52a-44e7-8ed9-0653e4185677 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:48.588378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:48.590241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:48.591201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 643/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 06:20:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:48.596410 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646d8a1-e060-4734-a5d6-3742921c7ab1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:48.692639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646d8a1-e060-4734-a5d6-3742921c7ab1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:48.725260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329f742-be34-4664-b61a-0f1c46703794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:48.725911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329f742-be34-4664-b61a-0f1c46703794 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:48.726231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329f742-be34-4664-b61a-0f1c46703794 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:48.726509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329f742-be34-4664-b61a-0f1c46703794 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:48.727214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329f742-be34-4664-b61a-0f1c46703794 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:48.727523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329f742-be34-4664-b61a-0f1c46703794 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:48.728885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329f742-be34-4664-b61a-0f1c46703794 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:48.734187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 648/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jul 21 06:20:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:48.739751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f4338-c52a-44e7-8ed9-0653e4185677 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:48.843906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f4338-c52a-44e7-8ed9-0653e4185677 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:48.880433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e1bcf-e141-4c02-81cd-e2ef0fb625b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:48.881106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e1bcf-e141-4c02-81cd-e2ef0fb625b4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:48.881393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e1bcf-e141-4c02-81cd-e2ef0fb625b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:48.881676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e1bcf-e141-4c02-81cd-e2ef0fb625b4 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:48.882429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e1bcf-e141-4c02-81cd-e2ef0fb625b4 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:48.882739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e1bcf-e141-4c02-81cd-e2ef0fb625b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:48.884114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e1bcf-e141-4c02-81cd-e2ef0fb625b4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:48.889217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 644/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jul 21 06:20:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:50.915001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147bcc6-4ceb-4960-bdb4-96c7310d9e29 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:50.915542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147bcc6-4ceb-4960-bdb4-96c7310d9e29 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:50.915542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147bcc6-4ceb-4960-bdb4-96c7310d9e29 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:50.916913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 649/1293] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:20:50.922165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc4caa-21d7-4ca5-aec2-aac7e26b86a0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:50.922598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc4caa-21d7-4ca5-aec2-aac7e26b86a0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:50.922735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc4caa-21d7-4ca5-aec2-aac7e26b86a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:50.939731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-16cc4caa-21d7-4ca5-aec2-aac7e26b86a0 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:20:50.940896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-16cc4caa-21d7-4ca5-aec2-aac7e26b86a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:20:50.969735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:51.006619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 645/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:20:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:20:51.029165 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147bcc6-4ceb-4960-bdb4-96c7310d9e29 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:51.061106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:51.130867 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147bcc6-4ceb-4960-bdb4-96c7310d9e29 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:51.163636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe8754-88ad-4ecc-8316-8613a9f57c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:51.164291 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe8754-88ad-4ecc-8316-8613a9f57c1d None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:51.164574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe8754-88ad-4ecc-8316-8613a9f57c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:51.164851 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe8754-88ad-4ecc-8316-8613a9f57c1d None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:51.165612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe8754-88ad-4ecc-8316-8613a9f57c1d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:51.167585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe8754-88ad-4ecc-8316-8613a9f57c1d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:51.179844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 650/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 06:20:51 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:20:53.195415 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b681340-fb78-448c-b194-f7e0efb9f823 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:53.195973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b681340-fb78-448c-b194-f7e0efb9f823 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:53.195973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b681340-fb78-448c-b194-f7e0efb9f823 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:53.197339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 646/1296] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:20:53.203110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940a7bb-48b2-4e46-aa50-5b13251571ad None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:53.203469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940a7bb-48b2-4e46-aa50-5b13251571ad None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:53.203666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940a7bb-48b2-4e46-aa50-5b13251571ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:53.220742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2940a7bb-48b2-4e46-aa50-5b13251571ad None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:20:53.221909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2940a7bb-48b2-4e46-aa50-5b13251571ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:20:53.292855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 651/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:20:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:20:53.315941 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b681340-fb78-448c-b194-f7e0efb9f823 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:53.416421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b681340-fb78-448c-b194-f7e0efb9f823 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:53.450864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aafe9-a0a0-4e8d-8bc4-7440596dacfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:53.451735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aafe9-a0a0-4e8d-8bc4-7440596dacfc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:53.452015 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aafe9-a0a0-4e8d-8bc4-7440596dacfc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:53.452287 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aafe9-a0a0-4e8d-8bc4-7440596dacfc None admin] PATH_INFO: `/v3/services/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:53.452931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1aafe9-a0a0-4e8d-8bc4-7440596dacfc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:53.454322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1aafe9-a0a0-4e8d-8bc4-7440596dacfc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4e1aafe9-a0a0-4e8d-8bc4-7440596dacfc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:53.466483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:53.467644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:53.468662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 647/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 06:20:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:53.473486 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940a7bb-48b2-4e46-aa50-5b13251571ad None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:53.493871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:53.575491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940a7bb-48b2-4e46-aa50-5b13251571ad None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:53.608547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f44b7-ce0d-468e-a512-59265924dbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:53.609202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f44b7-ce0d-468e-a512-59265924dbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:53.609486 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f44b7-ce0d-468e-a512-59265924dbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:53.609763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f44b7-ce0d-468e-a512-59265924dbb3 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:53.610567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f44b7-ce0d-468e-a512-59265924dbb3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:53.610878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f44b7-ce0d-468e-a512-59265924dbb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:53.612246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f44b7-ce0d-468e-a512-59265924dbb3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:53.617245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 652/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jul 21 06:20:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:53.623443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aafe9-a0a0-4e8d-8bc4-7440596dacfc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:53.654198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:53.719375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aafe9-a0a0-4e8d-8bc4-7440596dacfc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:53.751437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bc389-efbc-439a-858d-a90fda22df8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:53.752072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc389-efbc-439a-858d-a90fda22df8d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:53.752351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc389-efbc-439a-858d-a90fda22df8d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:53.752625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc389-efbc-439a-858d-a90fda22df8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:53.753404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bc389-efbc-439a-858d-a90fda22df8d None admin] RBAC: Adding query filter params (interface=public, service_id=76a518976150415d9d21ebeab8189898, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:53.753712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bc389-efbc-439a-858d-a90fda22df8d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:53.755085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bc389-efbc-439a-858d-a90fda22df8d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:53.759931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 648/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 06:20:53 2022] GET /identity/v3/endpoints?service_id=76a518976150415d9d21ebeab8189898&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:53.765541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f44b7-ce0d-468e-a512-59265924dbb3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:53.853033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:53.876477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f44b7-ce0d-468e-a512-59265924dbb3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:53.911593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a5851-471b-48a1-b0ab-6d3df2ade6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:53.912247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a5851-471b-48a1-b0ab-6d3df2ade6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:53.912527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a5851-471b-48a1-b0ab-6d3df2ade6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:53.912801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a5851-471b-48a1-b0ab-6d3df2ade6d0 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:53.913472 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a5851-471b-48a1-b0ab-6d3df2ade6d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:53.914857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a5851-471b-48a1-b0ab-6d3df2ade6d0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:53.922406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 653/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 06:20:53 2022] GET /identity/v3/services => generated 2191 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:20:55.936956 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8dc23-cfaa-4cee-973e-d87432672497 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:55.937510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8dc23-cfaa-4cee-973e-d87432672497 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:55.937612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8dc23-cfaa-4cee-973e-d87432672497 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:55.938936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 649/1302] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:20:55.943937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dba45-79db-475c-b5b6-939334f88ad8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:55.944377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dba45-79db-475c-b5b6-939334f88ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:55.944482 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dba45-79db-475c-b5b6-939334f88ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:55.960947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c78dba45-79db-475c-b5b6-939334f88ad8 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:20:55.962149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c78dba45-79db-475c-b5b6-939334f88ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:20:56.028063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 654/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:20:55 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 06:20:56.050790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8dc23-cfaa-4cee-973e-d87432672497 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:56.142698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:56.149905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8dc23-cfaa-4cee-973e-d87432672497 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:56.183457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff057663-890b-4987-a7f2-3b48b35eeceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:56.184241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff057663-890b-4987-a7f2-3b48b35eeceb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:56.184521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff057663-890b-4987-a7f2-3b48b35eeceb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:56.184795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff057663-890b-4987-a7f2-3b48b35eeceb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:56.185443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff057663-890b-4987-a7f2-3b48b35eeceb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:56.186810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff057663-890b-4987-a7f2-3b48b35eeceb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ff057663-890b-4987-a7f2-3b48b35eeceb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 06:20:56.199131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 06:20:56.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 21 06:20:56.201134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 650/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 06:20:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 06:20:56.206234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dba45-79db-475c-b5b6-939334f88ad8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:56.280862 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:20:56.303705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dba45-79db-475c-b5b6-939334f88ad8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:56.337045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d5363-85c7-4b79-8d4b-3432259474aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:56.337704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d5363-85c7-4b79-8d4b-3432259474aa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:56.338014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d5363-85c7-4b79-8d4b-3432259474aa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:56.338473 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d5363-85c7-4b79-8d4b-3432259474aa None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:56.339181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d5363-85c7-4b79-8d4b-3432259474aa None admin] RBAC: Adding query filter params (name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:20:56.339491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d5363-85c7-4b79-8d4b-3432259474aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:56.340859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d5363-85c7-4b79-8d4b-3432259474aa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:56.345885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 655/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jul 21 06:20:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:20:56.351495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff057663-890b-4987-a7f2-3b48b35eeceb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:20:56.448703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff057663-890b-4987-a7f2-3b48b35eeceb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:20:56.480660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb603dbe-6bf1-43c9-b8dc-87b1fc4d1e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:20:56.481295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb603dbe-6bf1-43c9-b8dc-87b1fc4d1e56 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:20:56.481578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb603dbe-6bf1-43c9-b8dc-87b1fc4d1e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:20:56.481856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb603dbe-6bf1-43c9-b8dc-87b1fc4d1e56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:20:56.482619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb603dbe-6bf1-43c9-b8dc-87b1fc4d1e56 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:20:56.483963 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb603dbe-6bf1-43c9-b8dc-87b1fc4d1e56 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:20:56.502006 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 651/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 06:20:56 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:21:06.342997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=30 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:21:06.497982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:21:59.236135 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9195ad-fee4-454d-8f7f-cb1ffe78fa30 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:21:59.236739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9195ad-fee4-454d-8f7f-cb1ffe78fa30 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:21:59.236739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9195ad-fee4-454d-8f7f-cb1ffe78fa30 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:21:59.238212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 656/1307] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:21:59.247443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60262565-8b00-4a3c-91a8-c62af1de2010 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:21:59.247949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60262565-8b00-4a3c-91a8-c62af1de2010 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:21:59.247949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60262565-8b00-4a3c-91a8-c62af1de2010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:21:59.423465 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-60262565-8b00-4a3c-91a8-c62af1de2010 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:21:59.424783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-60262565-8b00-4a3c-91a8-c62af1de2010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:21:59.663111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 652/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:21:59.717675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b15658-7fda-4c68-8aea-a4c87b9b8bb3 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:21:59.718317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b15658-7fda-4c68-8aea-a4c87b9b8bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:21:59.718892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b15658-7fda-4c68-8aea-a4c87b9b8bb3 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:21:59.721423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 657/1309] 10.222.0.42 () {54 vars in 965 bytes} [Thu Jul 21 06:21:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:21:59.733831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93908c1d-9c5a-43dd-990d-14cd0dec3a39 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:21:59.734166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93908c1d-9c5a-43dd-990d-14cd0dec3a39 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:21:59.734459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93908c1d-9c5a-43dd-990d-14cd0dec3a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:21:59.909327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-93908c1d-9c5a-43dd-990d-14cd0dec3a39 None None] MFA Rules not processed for user `1a9c3f34eaeb4b538528ff9050252dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:21:59.913055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-93908c1d-9c5a-43dd-990d-14cd0dec3a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:21:59.946553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:00.039153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 653/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Thu Jul 21 06:21:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:00.051184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b15658-7fda-4c68-8aea-a4c87b9b8bb3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:00.054241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f2b15658-7fda-4c68-8aea-a4c87b9b8bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:00.194023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b15658-7fda-4c68-8aea-a4c87b9b8bb3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:00.280106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0ebfe-2fef-440c-9a63-2a60ea4dc1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:00.281490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0ebfe-2fef-440c-9a63-2a60ea4dc1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:00.281931 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0ebfe-2fef-440c-9a63-2a60ea4dc1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:00.282379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0ebfe-2fef-440c-9a63-2a60ea4dc1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:00.284319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a3d0ebfe-2fef-440c-9a63-2a60ea4dc1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:00.345931 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:00.413986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0ebfe-2fef-440c-9a63-2a60ea4dc1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:00.413986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0ebfe-2fef-440c-9a63-2a60ea4dc1e3 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:00.433047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 658/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:22:02.901045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc81e4-8acc-46ae-9d6e-27df7da6c6a7 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:02.901599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc81e4-8acc-46ae-9d6e-27df7da6c6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:02.901599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc81e4-8acc-46ae-9d6e-27df7da6c6a7 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:02.903024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 654/1312] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:02.908441 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1b49b-b58a-4bf7-9e3a-25ffb7efb4d4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:02.908829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1b49b-b58a-4bf7-9e3a-25ffb7efb4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:02.909034 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1b49b-b58a-4bf7-9e3a-25ffb7efb4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:02.989652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d8a1b49b-b58a-4bf7-9e3a-25ffb7efb4d4 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:02.991085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d8a1b49b-b58a-4bf7-9e3a-25ffb7efb4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:03.110155 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 659/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:03.155702 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc81e4-8acc-46ae-9d6e-27df7da6c6a7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:03.157520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8fdc81e4-8acc-46ae-9d6e-27df7da6c6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:03.276511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc81e4-8acc-46ae-9d6e-27df7da6c6a7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:03.351347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:03.353868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f837fc6-0b9d-474d-b5a1-1f9bc0c44345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:03.354574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f837fc6-0b9d-474d-b5a1-1f9bc0c44345 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:03.354804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f837fc6-0b9d-474d-b5a1-1f9bc0c44345 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:03.355082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f837fc6-0b9d-474d-b5a1-1f9bc0c44345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:03.357567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9f837fc6-0b9d-474d-b5a1-1f9bc0c44345 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:03.470848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f837fc6-0b9d-474d-b5a1-1f9bc0c44345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:03.472292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f837fc6-0b9d-474d-b5a1-1f9bc0c44345 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:03.491269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 655/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:05.852984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57ed5c-54c8-44cc-8a2e-f8114fc9282a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:05.853518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57ed5c-54c8-44cc-8a2e-f8114fc9282a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:05.853638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57ed5c-54c8-44cc-8a2e-f8114fc9282a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:05.855024 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 660/1315] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:05.860347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2e4fe-71ab-49d3-9ac7-a7381e771596 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:05.860759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2e4fe-71ab-49d3-9ac7-a7381e771596 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:05.860915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2e4fe-71ab-49d3-9ac7-a7381e771596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:05.924504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-27b2e4fe-71ab-49d3-9ac7-a7381e771596 None None] MFA Rules not processed for user `d68b999da6f749cd98aa5b80062e64f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:05.925693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-27b2e4fe-71ab-49d3-9ac7-a7381e771596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:05.978217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:06.023019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 656/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:06.063843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57ed5c-54c8-44cc-8a2e-f8114fc9282a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:06.065385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1a57ed5c-54c8-44cc-8a2e-f8114fc9282a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:06.075305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:06.169090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57ed5c-54c8-44cc-8a2e-f8114fc9282a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:06.245149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f4053-be6d-4578-abcf-06469fa04455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:06.245787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f4053-be6d-4578-abcf-06469fa04455 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:06.246101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f4053-be6d-4578-abcf-06469fa04455 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:06.246378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f4053-be6d-4578-abcf-06469fa04455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:06.247990 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-061f4053-be6d-4578-abcf-06469fa04455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:06.309711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:06.357504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f4053-be6d-4578-abcf-06469fa04455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:06.358964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f4053-be6d-4578-abcf-06469fa04455 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:06.377393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 661/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:08.993049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5802fe-48a4-48cf-9328-a58b4c35736a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:08.993590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5802fe-48a4-48cf-9328-a58b4c35736a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:08.993590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5802fe-48a4-48cf-9328-a58b4c35736a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:08.995028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 657/1318] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:09.000044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1d755-b550-4286-b490-8656d636c400 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:09.000484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1d755-b550-4286-b490-8656d636c400 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:09.000605 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1d755-b550-4286-b490-8656d636c400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:09.066243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e9b1d755-b550-4286-b490-8656d636c400 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:09.067426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e9b1d755-b550-4286-b490-8656d636c400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:09.164814 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 662/1319] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:09.203491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5802fe-48a4-48cf-9328-a58b4c35736a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:09.205030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8e5802fe-48a4-48cf-9328-a58b4c35736a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:09.308374 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5802fe-48a4-48cf-9328-a58b4c35736a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:09.381464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c3606-56d8-4c9d-b40d-944058a6f512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:09.382149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c3606-56d8-4c9d-b40d-944058a6f512 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:09.382436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c3606-56d8-4c9d-b40d-944058a6f512 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:09.382715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c3606-56d8-4c9d-b40d-944058a6f512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:09.384336 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ca0c3606-56d8-4c9d-b40d-944058a6f512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:09.392017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:09.493081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c3606-56d8-4c9d-b40d-944058a6f512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:09.494543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c3606-56d8-4c9d-b40d-944058a6f512 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:09.512631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 658/1320] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:12.213353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3f8c8-6b79-4b65-89c5-2871d28e59d6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:12.213922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3f8c8-6b79-4b65-89c5-2871d28e59d6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:12.213922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3f8c8-6b79-4b65-89c5-2871d28e59d6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:12.215739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 663/1321] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:12.221645 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2335b4-9eac-415e-9d10-aa3f62d8d19f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:12.222128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2335b4-9eac-415e-9d10-aa3f62d8d19f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:12.222238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2335b4-9eac-415e-9d10-aa3f62d8d19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:12.297441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5d2335b4-9eac-415e-9d10-aa3f62d8d19f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:12.298678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5d2335b4-9eac-415e-9d10-aa3f62d8d19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:12.369193 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:12.414037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 659/1322] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:12.450263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3f8c8-6b79-4b65-89c5-2871d28e59d6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:12.451816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-40c3f8c8-6b79-4b65-89c5-2871d28e59d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:12.472649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:12.563354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3f8c8-6b79-4b65-89c5-2871d28e59d6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:12.636776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe0da5-5da9-4998-a0ae-16ec0db8cac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:12.637336 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0da5-5da9-4998-a0ae-16ec0db8cac9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:12.637617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0da5-5da9-4998-a0ae-16ec0db8cac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:12.637894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0da5-5da9-4998-a0ae-16ec0db8cac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:12.639544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9ebe0da5-5da9-4998-a0ae-16ec0db8cac9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:12.713232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:12.753423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe0da5-5da9-4998-a0ae-16ec0db8cac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:12.754898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe0da5-5da9-4998-a0ae-16ec0db8cac9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:12.773672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 664/1323] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:14.906750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9ce8e-d84e-4fba-8938-ae61668162b7 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:14.907352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9ce8e-d84e-4fba-8938-ae61668162b7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:14.907352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9ce8e-d84e-4fba-8938-ae61668162b7 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:14.908565 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 660/1324] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:14.914417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b62de-b530-4085-87b2-7462ca870138 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:14.914890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b62de-b530-4085-87b2-7462ca870138 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:14.914996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b62de-b530-4085-87b2-7462ca870138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:14.980145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-738b62de-b530-4085-87b2-7462ca870138 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:14.981332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-738b62de-b530-4085-87b2-7462ca870138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:15.080273 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 665/1325] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:15.116808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9ce8e-d84e-4fba-8938-ae61668162b7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:15.118382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-34b9ce8e-d84e-4fba-8938-ae61668162b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:15.224998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9ce8e-d84e-4fba-8938-ae61668162b7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:15.303012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd508-5c0f-4c07-9e51-400ae0dd469d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:15.303638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd508-5c0f-4c07-9e51-400ae0dd469d service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:15.303925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd508-5c0f-4c07-9e51-400ae0dd469d service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:15.304204 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd508-5c0f-4c07-9e51-400ae0dd469d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:15.305846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-279dd508-5c0f-4c07-9e51-400ae0dd469d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:15.322936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:15.418803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279dd508-5c0f-4c07-9e51-400ae0dd469d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:15.420237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279dd508-5c0f-4c07-9e51-400ae0dd469d service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:15.439377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 661/1326] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:17.547800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566308e7-876a-4244-8d43-fb3cefb3bfb5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:17.548348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566308e7-876a-4244-8d43-fb3cefb3bfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:17.548348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566308e7-876a-4244-8d43-fb3cefb3bfb5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:17.549882 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 666/1327] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:17.555562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2dde1-d587-4b77-b9e5-7cf797f86309 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:17.555999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2dde1-d587-4b77-b9e5-7cf797f86309 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:17.556130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2dde1-d587-4b77-b9e5-7cf797f86309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:17.624360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-72c2dde1-d587-4b77-b9e5-7cf797f86309 None None] MFA Rules not processed for user `d68b999da6f749cd98aa5b80062e64f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:17.625549 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-72c2dde1-d587-4b77-b9e5-7cf797f86309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:17.692948 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:17.726215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 662/1328] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:17 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:17.764941 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566308e7-876a-4244-8d43-fb3cefb3bfb5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:17.766581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-566308e7-876a-4244-8d43-fb3cefb3bfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:17.798256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:17.876737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566308e7-876a-4244-8d43-fb3cefb3bfb5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:17.952261 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdf752-080c-4cea-bf80-334f709af716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:17.952909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdf752-080c-4cea-bf80-334f709af716 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:17.953198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdf752-080c-4cea-bf80-334f709af716 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:17.953478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdf752-080c-4cea-bf80-334f709af716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:17.955146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2ebdf752-080c-4cea-bf80-334f709af716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:18.033516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:18.083862 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdf752-080c-4cea-bf80-334f709af716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:18.085351 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdf752-080c-4cea-bf80-334f709af716 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:18.107756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 667/1329] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:17 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:20.406864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadd17b-f53b-4f0c-b2d8-99c0618ac714 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:20.407420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadd17b-f53b-4f0c-b2d8-99c0618ac714 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:20.407420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadd17b-f53b-4f0c-b2d8-99c0618ac714 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:20.408808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 663/1330] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:20.414693 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc3bfd-25cc-4ede-a4e2-430e7c96a60a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:20.415120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc3bfd-25cc-4ede-a4e2-430e7c96a60a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:20.415255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc3bfd-25cc-4ede-a4e2-430e7c96a60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:20.479186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-17cc3bfd-25cc-4ede-a4e2-430e7c96a60a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:20.480383 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-17cc3bfd-25cc-4ede-a4e2-430e7c96a60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:20.575215 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 668/1331] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:20.611918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadd17b-f53b-4f0c-b2d8-99c0618ac714 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:20.613469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7eadd17b-f53b-4f0c-b2d8-99c0618ac714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:20.714910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadd17b-f53b-4f0c-b2d8-99c0618ac714 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:20.786522 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21b960-e665-4609-a763-33073adc0577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:20.787186 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21b960-e665-4609-a763-33073adc0577 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:20.787449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21b960-e665-4609-a763-33073adc0577 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:20.787726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21b960-e665-4609-a763-33073adc0577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:20.789332 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9a21b960-e665-4609-a763-33073adc0577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:20.815453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:20.898559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21b960-e665-4609-a763-33073adc0577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:20.899998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21b960-e665-4609-a763-33073adc0577 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:20.918744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 664/1332] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:23.040653 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a8bf7-e7fe-4a1a-a866-8fec6a73db9c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:23.041222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a8bf7-e7fe-4a1a-a866-8fec6a73db9c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:23.041222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a8bf7-e7fe-4a1a-a866-8fec6a73db9c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:23.042632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 669/1333] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:23.047997 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e0c0d-9b19-44c2-8b20-7fbc7e6eb951 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:23.048427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e0c0d-9b19-44c2-8b20-7fbc7e6eb951 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:23.048564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e0c0d-9b19-44c2-8b20-7fbc7e6eb951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:23.113522 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cb9e0c0d-9b19-44c2-8b20-7fbc7e6eb951 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:23.114833 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cb9e0c0d-9b19-44c2-8b20-7fbc7e6eb951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:23.188289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:23.211973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 665/1334] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:23.250920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a8bf7-e7fe-4a1a-a866-8fec6a73db9c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:23.252438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b71a8bf7-e7fe-4a1a-a866-8fec6a73db9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:23.293929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:23.357189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a8bf7-e7fe-4a1a-a866-8fec6a73db9c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:23.436147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8b16c-30d3-436a-86a5-b37ef29acae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:23.436794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8b16c-30d3-436a-86a5-b37ef29acae0 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:23.437074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8b16c-30d3-436a-86a5-b37ef29acae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:23.437344 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8b16c-30d3-436a-86a5-b37ef29acae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:23.438994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5cb8b16c-30d3-436a-86a5-b37ef29acae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:23.520114 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:23.546326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8b16c-30d3-436a-86a5-b37ef29acae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:23.547703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8b16c-30d3-436a-86a5-b37ef29acae0 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:23.566416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 670/1335] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:25.947644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806e252-7f9d-4917-b1b9-74dbcfdeb902 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:25.948181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806e252-7f9d-4917-b1b9-74dbcfdeb902 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:25.948181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806e252-7f9d-4917-b1b9-74dbcfdeb902 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:25.949585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 666/1336] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:22:25.954668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335bbf7-3412-4067-a12f-6d5892ee3a64 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:25.955108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335bbf7-3412-4067-a12f-6d5892ee3a64 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:25.955238 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335bbf7-3412-4067-a12f-6d5892ee3a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:26.021322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c335bbf7-3412-4067-a12f-6d5892ee3a64 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:26.022529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c335bbf7-3412-4067-a12f-6d5892ee3a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:26.118230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 671/1337] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:26.170677 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed1341-b06f-4251-bdc8-b57f5ee7de1e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:26.170962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed1341-b06f-4251-bdc8-b57f5ee7de1e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:26.171242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed1341-b06f-4251-bdc8-b57f5ee7de1e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:26.172629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 667/1338] 10.222.0.42 () {54 vars in 965 bytes} [Thu Jul 21 06:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:26.179393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82b62a-e431-4056-b71f-b30896a4a3db None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:26.179696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82b62a-e431-4056-b71f-b30896a4a3db None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:26.179978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82b62a-e431-4056-b71f-b30896a4a3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:26.233153 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:26.255379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5d82b62a-e431-4056-b71f-b30896a4a3db None None] MFA Rules not processed for user `1a9c3f34eaeb4b538528ff9050252dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:26.256577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5d82b62a-e431-4056-b71f-b30896a4a3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:26.330861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 672/1339] 10.222.0.42 () {58 vars in 1062 bytes} [Thu Jul 21 06:22:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:26.339637 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed1341-b06f-4251-bdc8-b57f5ee7de1e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:26.341156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-34ed1341-b06f-4251-bdc8-b57f5ee7de1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:26.444062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed1341-b06f-4251-bdc8-b57f5ee7de1e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:26.515714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2776f24-fb30-4d0f-ab62-d9e30f79606d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:26.516327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2776f24-fb30-4d0f-ab62-d9e30f79606d service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:26.516615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2776f24-fb30-4d0f-ab62-d9e30f79606d service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:26.516902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2776f24-fb30-4d0f-ab62-d9e30f79606d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:26.518553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b2776f24-fb30-4d0f-ab62-d9e30f79606d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:26.553823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:26.626304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2776f24-fb30-4d0f-ab62-d9e30f79606d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:26.627747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2776f24-fb30-4d0f-ab62-d9e30f79606d service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:26.646939 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 668/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:31.187396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ff3fd-7203-425b-9d71-d049e56ab7d2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:31.187900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ff3fd-7203-425b-9d71-d049e56ab7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:31.187900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ff3fd-7203-425b-9d71-d049e56ab7d2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:31.189245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 673/1341] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:31.194883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bc3ac-007d-488e-8c8e-9e8b86619c2c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:31.195394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bc3ac-007d-488e-8c8e-9e8b86619c2c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:31.195491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bc3ac-007d-488e-8c8e-9e8b86619c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:31.261713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ba1bc3ac-007d-488e-8c8e-9e8b86619c2c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:31.262945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ba1bc3ac-007d-488e-8c8e-9e8b86619c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:31.348221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:31.360657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 669/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:22:31.401206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ff3fd-7203-425b-9d71-d049e56ab7d2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:31.402849 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5d5ff3fd-7203-425b-9d71-d049e56ab7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:31.510797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ff3fd-7203-425b-9d71-d049e56ab7d2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:31.548353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:31.585303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255cdeb-c149-4b94-b567-65a8005d7962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:31.586016 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255cdeb-c149-4b94-b567-65a8005d7962 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:31.586303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255cdeb-c149-4b94-b567-65a8005d7962 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:31.586581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255cdeb-c149-4b94-b567-65a8005d7962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:31.588225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0255cdeb-c149-4b94-b567-65a8005d7962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:31.703965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255cdeb-c149-4b94-b567-65a8005d7962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:31.705452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255cdeb-c149-4b94-b567-65a8005d7962 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:31.724761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 674/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:34.442437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0e9d-57f6-4abe-994e-0b94d946d45e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:34.442856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0e9d-57f6-4abe-994e-0b94d946d45e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:34.443028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0e9d-57f6-4abe-994e-0b94d946d45e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:34.445625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 670/1344] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:34.450989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04c591-1e2b-4eac-8277-7c151087efea None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:34.451430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04c591-1e2b-4eac-8277-7c151087efea None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:34.451540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04c591-1e2b-4eac-8277-7c151087efea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:34.517957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0e04c591-1e2b-4eac-8277-7c151087efea None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:34.519162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0e04c591-1e2b-4eac-8277-7c151087efea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:34.537984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:34.616989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 675/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:34.655262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed0e9d-57f6-4abe-994e-0b94d946d45e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:34.656811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c6ed0e9d-57f6-4abe-994e-0b94d946d45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:34.761740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed0e9d-57f6-4abe-994e-0b94d946d45e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:34.838065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e8d4d-9000-4812-8fc8-0bf47d69b0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:34.838734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8d4d-9000-4812-8fc8-0bf47d69b0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:34.839017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8d4d-9000-4812-8fc8-0bf47d69b0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:34.839307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8d4d-9000-4812-8fc8-0bf47d69b0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:34.841225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-607e8d4d-9000-4812-8fc8-0bf47d69b0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:34.891697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:34.958594 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e8d4d-9000-4812-8fc8-0bf47d69b0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:34.960042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e8d4d-9000-4812-8fc8-0bf47d69b0a1 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:34.978904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 671/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:38.990924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e964ef6-6359-4dfd-9a09-0188eea3a0fa None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:38.991456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e964ef6-6359-4dfd-9a09-0188eea3a0fa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:38.991456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e964ef6-6359-4dfd-9a09-0188eea3a0fa None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:38.992831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 676/1347] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:38.998254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944651b-dcb7-4fa8-8e4a-402bf0b85ce3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:38.998663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944651b-dcb7-4fa8-8e4a-402bf0b85ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:38.998827 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944651b-dcb7-4fa8-8e4a-402bf0b85ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:39.066826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a944651b-dcb7-4fa8-8e4a-402bf0b85ce3 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:39.068024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a944651b-dcb7-4fa8-8e4a-402bf0b85ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:39.168633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:39.171842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 672/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:39.212297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e964ef6-6359-4dfd-9a09-0188eea3a0fa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:39.213909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8e964ef6-6359-4dfd-9a09-0188eea3a0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:39.336119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e964ef6-6359-4dfd-9a09-0188eea3a0fa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:39.386102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:39.420699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3b6f8-7b25-4fbf-9353-4bca4c2c035f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:39.421553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3b6f8-7b25-4fbf-9353-4bca4c2c035f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:39.421668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3b6f8-7b25-4fbf-9353-4bca4c2c035f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:39.421909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3b6f8-7b25-4fbf-9353-4bca4c2c035f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:39.423611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d1a3b6f8-7b25-4fbf-9353-4bca4c2c035f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:39.554449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3b6f8-7b25-4fbf-9353-4bca4c2c035f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:39.556170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3b6f8-7b25-4fbf-9353-4bca4c2c035f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:39.578998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 677/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:44.448170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d7f01-d12a-4e26-8c34-76a67e903c33 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:44.448691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d7f01-d12a-4e26-8c34-76a67e903c33 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:44.449239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d7f01-d12a-4e26-8c34-76a67e903c33 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:44.451787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 673/1350] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 06:22:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:44.458577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0970b98-d78e-4463-9884-5b68a45dae2e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:44.458986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0970b98-d78e-4463-9884-5b68a45dae2e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:44.459198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0970b98-d78e-4463-9884-5b68a45dae2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:44.574742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d0970b98-d78e-4463-9884-5b68a45dae2e None None] MFA Rules not processed for user `1da2d85eb18049f196aeefc51d3f1e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:44.576377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d0970b98-d78e-4463-9884-5b68a45dae2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:44.597447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:44.702987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 678/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 06:22:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:48.765524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5949697-6461-4b15-939f-17a594d4fcab None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:48.766116 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5949697-6461-4b15-939f-17a594d4fcab None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:48.766116 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5949697-6461-4b15-939f-17a594d4fcab None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:48.767756 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 674/1352] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 06:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:48.774707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02833a13-87e5-400e-bd77-41a715b05ec3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:48.775161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02833a13-87e5-400e-bd77-41a715b05ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:48.775268 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02833a13-87e5-400e-bd77-41a715b05ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:48.853293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-02833a13-87e5-400e-bd77-41a715b05ec3 None None] MFA Rules not processed for user `1da2d85eb18049f196aeefc51d3f1e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:48.854518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-02833a13-87e5-400e-bd77-41a715b05ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:48.929759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:48.969231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 679/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 06:22:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:50.456053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032798f-daf7-4fba-8ebe-93393f877899 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:50.456532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032798f-daf7-4fba-8ebe-93393f877899 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:50.456532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032798f-daf7-4fba-8ebe-93393f877899 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:50.457955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 675/1354] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 06:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:50.463048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc775a-a5ba-49a6-9972-d557bf2d1664 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:50.463505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc775a-a5ba-49a6-9972-d557bf2d1664 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:50.463723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc775a-a5ba-49a6-9972-d557bf2d1664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:50.543735 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d0dc775a-a5ba-49a6-9972-d557bf2d1664 None None] MFA Rules not processed for user `1da2d85eb18049f196aeefc51d3f1e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:50.544923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d0dc775a-a5ba-49a6-9972-d557bf2d1664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:50.658869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 680/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 06:22:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:54.188514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00f8c3-5ebb-4dc9-bf50-dabe2a6bd843 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:54.189518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00f8c3-5ebb-4dc9-bf50-dabe2a6bd843 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:54.190070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00f8c3-5ebb-4dc9-bf50-dabe2a6bd843 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:54.192254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 676/1356] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:22:54.198539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a003cb9-2bd9-4f15-9d10-767fe84fc641 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:54.199418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a003cb9-2bd9-4f15-9d10-767fe84fc641 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:54.199937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a003cb9-2bd9-4f15-9d10-767fe84fc641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:54.241233 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:54.277980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0a003cb9-2bd9-4f15-9d10-767fe84fc641 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:54.279788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0a003cb9-2bd9-4f15-9d10-767fe84fc641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:54.389071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 681/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:54.446081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99f47d-5b80-4ac9-bee8-c353ce4b815f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:54.446900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99f47d-5b80-4ac9-bee8-c353ce4b815f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:54.447417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99f47d-5b80-4ac9-bee8-c353ce4b815f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:54.449414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 677/1358] 10.222.0.42 () {54 vars in 962 bytes} [Thu Jul 21 06:22:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:54.461913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0f264-51eb-483b-a6b8-b3e9dbcd2a42 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:54.462243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0f264-51eb-483b-a6b8-b3e9dbcd2a42 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:54.462445 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0f264-51eb-483b-a6b8-b3e9dbcd2a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:54.550220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e1a0f264-51eb-483b-a6b8-b3e9dbcd2a42 None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:54.551982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e1a0f264-51eb-483b-a6b8-b3e9dbcd2a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:54.603883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:54.677792 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 682/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Jul 21 06:22:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:54.685554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99f47d-5b80-4ac9-bee8-c353ce4b815f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:54.687105 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1a99f47d-5b80-4ac9-bee8-c353ce4b815f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:54.843104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99f47d-5b80-4ac9-bee8-c353ce4b815f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:54.918506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:54.968381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241da20f-f8e8-4058-91af-cbfc46b2ec12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:54.969037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241da20f-f8e8-4058-91af-cbfc46b2ec12 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:54.969324 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241da20f-f8e8-4058-91af-cbfc46b2ec12 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:54.969601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241da20f-f8e8-4058-91af-cbfc46b2ec12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:54.971258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-241da20f-f8e8-4058-91af-cbfc46b2ec12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:55.072803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241da20f-f8e8-4058-91af-cbfc46b2ec12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:55.074440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241da20f-f8e8-4058-91af-cbfc46b2ec12 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:55.095832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 678/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:22:57.375448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571637b9-5454-489c-8133-e5826e9fbc36 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:57.376189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571637b9-5454-489c-8133-e5826e9fbc36 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:57.376189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571637b9-5454-489c-8133-e5826e9fbc36 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:57.377567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 683/1361] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:22:57.383914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf7b90-371b-4a9d-82da-864b0f61240c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:57.384448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf7b90-371b-4a9d-82da-864b0f61240c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:57.384448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf7b90-371b-4a9d-82da-864b0f61240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:57.456348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-29bf7b90-371b-4a9d-82da-864b0f61240c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:22:57.458384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-29bf7b90-371b-4a9d-82da-864b0f61240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:57.497689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:57.563817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 679/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:22:57.611382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571637b9-5454-489c-8133-e5826e9fbc36 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:22:57.613070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-571637b9-5454-489c-8133-e5826e9fbc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:57.797276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571637b9-5454-489c-8133-e5826e9fbc36 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:22:57.806917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:22:57.949050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439366b-d295-4ebf-a29f-d1104156e849 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:22:57.949720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439366b-d295-4ebf-a29f-d1104156e849 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:22:57.950045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439366b-d295-4ebf-a29f-d1104156e849 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:22:57.950334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439366b-d295-4ebf-a29f-d1104156e849 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:22:57.951998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1439366b-d295-4ebf-a29f-d1104156e849 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:22:58.049729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439366b-d295-4ebf-a29f-d1104156e849 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:22:58.052857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439366b-d295-4ebf-a29f-d1104156e849 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:22:58.073774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 684/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:00.505035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c865e-2faa-488f-b0cd-5e4eb5c24832 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:00.505598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c865e-2faa-488f-b0cd-5e4eb5c24832 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:00.505700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c865e-2faa-488f-b0cd-5e4eb5c24832 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:00.507235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 680/1364] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:00.512533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf342-0602-433a-96d2-aa9282110248 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:00.512991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf342-0602-433a-96d2-aa9282110248 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:00.513289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf342-0602-433a-96d2-aa9282110248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:00.529103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:00.592049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-678cf342-0602-433a-96d2-aa9282110248 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:00.593380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-678cf342-0602-433a-96d2-aa9282110248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:00.710038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 685/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:00.752908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddbff4-abfa-4d7d-8012-8b5330a13c97 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:00.753257 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddbff4-abfa-4d7d-8012-8b5330a13c97 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:00.753756 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddbff4-abfa-4d7d-8012-8b5330a13c97 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:00.755308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 681/1366] 10.222.0.42 () {54 vars in 962 bytes} [Thu Jul 21 06:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:00.760951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09462750-4451-4e41-8b9a-87e02f89aabe None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:00.761314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09462750-4451-4e41-8b9a-87e02f89aabe None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:00.761821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09462750-4451-4e41-8b9a-87e02f89aabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:00.848525 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-09462750-4451-4e41-8b9a-87e02f89aabe None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:00.849814 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-09462750-4451-4e41-8b9a-87e02f89aabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:00.891082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:00.979742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 686/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Jul 21 06:23:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:00.987651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ddbff4-abfa-4d7d-8012-8b5330a13c97 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:00.989645 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-56ddbff4-abfa-4d7d-8012-8b5330a13c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:01.159806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ddbff4-abfa-4d7d-8012-8b5330a13c97 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:01.181499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:01.280555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34dda6-58a0-4cdd-9d92-f99f528b369d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:01.281223 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34dda6-58a0-4cdd-9d92-f99f528b369d service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:01.281511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34dda6-58a0-4cdd-9d92-f99f528b369d service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:01.281798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34dda6-58a0-4cdd-9d92-f99f528b369d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:01.283487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0d34dda6-58a0-4cdd-9d92-f99f528b369d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:01.378807 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34dda6-58a0-4cdd-9d92-f99f528b369d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:01.381074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34dda6-58a0-4cdd-9d92-f99f528b369d service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:01.400655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 682/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:02.473999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c5d47-572f-4a57-b3cf-c18f43aef9e6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:02.474543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c5d47-572f-4a57-b3cf-c18f43aef9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:02.474649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c5d47-572f-4a57-b3cf-c18f43aef9e6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:02.477993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 687/1369] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:02.481620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13ae82-f389-446a-aaba-e0fcdecd4dab None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:02.482106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13ae82-f389-446a-aaba-e0fcdecd4dab None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:02.482497 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13ae82-f389-446a-aaba-e0fcdecd4dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:02.516402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:02.556428 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5c13ae82-f389-446a-aaba-e0fcdecd4dab None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:02.557786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5c13ae82-f389-446a-aaba-e0fcdecd4dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:02.666994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 683/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:03.012082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5950b87-f570-4b5c-84bb-cd6a31556c74 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:03.012374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5950b87-f570-4b5c-84bb-cd6a31556c74 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:03.013370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5950b87-f570-4b5c-84bb-cd6a31556c74 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:03.014187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 688/1371] 10.222.0.42 () {54 vars in 964 bytes} [Thu Jul 21 06:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:03.019136 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a154878-b083-47ec-ba25-a9446b921b7f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:03.019433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a154878-b083-47ec-ba25-a9446b921b7f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:03.020583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a154878-b083-47ec-ba25-a9446b921b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:03.108838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9a154878-b083-47ec-ba25-a9446b921b7f None None] MFA Rules not processed for user `710c422b8024403da666ae5054ba1289`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:03.110068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9a154878-b083-47ec-ba25-a9446b921b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:03.164335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:03.196565 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 684/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Jul 21 06:23:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:03.204078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5950b87-f570-4b5c-84bb-cd6a31556c74 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:03.205591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d5950b87-f570-4b5c-84bb-cd6a31556c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:03.320268 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5950b87-f570-4b5c-84bb-cd6a31556c74 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:03.368477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:03.406178 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098def50-cd8c-45a4-b134-685de05a6bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:03.406697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098def50-cd8c-45a4-b134-685de05a6bbe service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:03.406981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098def50-cd8c-45a4-b134-685de05a6bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:03.407265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098def50-cd8c-45a4-b134-685de05a6bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:03.408923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-098def50-cd8c-45a4-b134-685de05a6bbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:03.528366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098def50-cd8c-45a4-b134-685de05a6bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:03.529820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098def50-cd8c-45a4-b134-685de05a6bbe service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:03.549468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 689/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:03.604960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd193be-dd7e-47eb-afcc-de0c70553994 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:03.605423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd193be-dd7e-47eb-afcc-de0c70553994 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:03.605519 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd193be-dd7e-47eb-afcc-de0c70553994 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:03.607323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 685/1374] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:03.612294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc63fd-2154-4f5d-a921-6e551ffa001a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:03.612589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc63fd-2154-4f5d-a921-6e551ffa001a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:03.612868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc63fd-2154-4f5d-a921-6e551ffa001a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:03.688472 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-54cc63fd-2154-4f5d-a921-6e551ffa001a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:03.689690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-54cc63fd-2154-4f5d-a921-6e551ffa001a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:03.718029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:03.817979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 690/1375] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:03.867714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd193be-dd7e-47eb-afcc-de0c70553994 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:03.869245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5bd193be-dd7e-47eb-afcc-de0c70553994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:04.028142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd193be-dd7e-47eb-afcc-de0c70553994 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:04.090154 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:04.155109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bdf33-4643-4aec-87a2-015dcc4dabde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:04.155769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bdf33-4643-4aec-87a2-015dcc4dabde service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:04.156059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bdf33-4643-4aec-87a2-015dcc4dabde service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:04.156342 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bdf33-4643-4aec-87a2-015dcc4dabde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:04.158034 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-be6bdf33-4643-4aec-87a2-015dcc4dabde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:04.250366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bdf33-4643-4aec-87a2-015dcc4dabde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:04.251981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bdf33-4643-4aec-87a2-015dcc4dabde service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:04.270704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 686/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:06.345350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45652c-3ccd-443f-96a9-2c3df0024c9d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:06.345826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45652c-3ccd-443f-96a9-2c3df0024c9d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:06.345924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45652c-3ccd-443f-96a9-2c3df0024c9d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:06.347333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 691/1377] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:06.353354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840a3d1-d486-4b57-8033-dfb3c19d24b6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:06.353782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840a3d1-d486-4b57-8033-dfb3c19d24b6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:06.353919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840a3d1-d486-4b57-8033-dfb3c19d24b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:06.419840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e840a3d1-d486-4b57-8033-dfb3c19d24b6 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:06.421028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e840a3d1-d486-4b57-8033-dfb3c19d24b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:06.428272 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:06.543449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 687/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:06.592471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45652c-3ccd-443f-96a9-2c3df0024c9d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:06.594136 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6e45652c-3ccd-443f-96a9-2c3df0024c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:06.743980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45652c-3ccd-443f-96a9-2c3df0024c9d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:06.747573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:06.868728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1152b-b4c1-4d8f-b512-782ff8b36856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:06.869389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1152b-b4c1-4d8f-b512-782ff8b36856 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:06.869677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1152b-b4c1-4d8f-b512-782ff8b36856 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:06.869981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1152b-b4c1-4d8f-b512-782ff8b36856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:06.871630 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-baf1152b-b4c1-4d8f-b512-782ff8b36856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:06.973455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1152b-b4c1-4d8f-b512-782ff8b36856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:06.975057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1152b-b4c1-4d8f-b512-782ff8b36856 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:07.002941 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 692/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:09.115323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104227d-5dde-4dda-89e2-99ab894309fc None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:09.115943 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104227d-5dde-4dda-89e2-99ab894309fc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:09.115943 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104227d-5dde-4dda-89e2-99ab894309fc None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:09.117673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 688/1380] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:09.123604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926f434-7312-4fcf-869a-8e45f0d93a3b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:09.124162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926f434-7312-4fcf-869a-8e45f0d93a3b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:09.124162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926f434-7312-4fcf-869a-8e45f0d93a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:09.136259 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:09.196981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a926f434-7312-4fcf-869a-8e45f0d93a3b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:09.198194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a926f434-7312-4fcf-869a-8e45f0d93a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:09.301630 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 693/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:09.344128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104227d-5dde-4dda-89e2-99ab894309fc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:09.345864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5104227d-5dde-4dda-89e2-99ab894309fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:09.504860 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:09.516648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104227d-5dde-4dda-89e2-99ab894309fc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:09.638832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce50b1-95fb-4a99-8cab-f2661ce1ba60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:09.639658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce50b1-95fb-4a99-8cab-f2661ce1ba60 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:09.639948 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce50b1-95fb-4a99-8cab-f2661ce1ba60 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:09.640225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce50b1-95fb-4a99-8cab-f2661ce1ba60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:09.641905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7cce50b1-95fb-4a99-8cab-f2661ce1ba60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:09.740604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce50b1-95fb-4a99-8cab-f2661ce1ba60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:09.742268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce50b1-95fb-4a99-8cab-f2661ce1ba60 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:09.753205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:09.763228 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 689/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:11.872125 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d3b24-2b13-40a1-99c6-6812c82a0fa6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:11.872655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d3b24-2b13-40a1-99c6-6812c82a0fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:11.872655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d3b24-2b13-40a1-99c6-6812c82a0fa6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:11.874072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 694/1383] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:11.880275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819c022-d0d7-4048-afca-179e2e810860 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:11.880275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819c022-d0d7-4048-afca-179e2e810860 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:11.881113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819c022-d0d7-4048-afca-179e2e810860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:11.949693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8819c022-d0d7-4048-afca-179e2e810860 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:11.950877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8819c022-d0d7-4048-afca-179e2e810860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:12.056961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 690/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:12.098734 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d3b24-2b13-40a1-99c6-6812c82a0fa6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:12.100359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cc2d3b24-2b13-40a1-99c6-6812c82a0fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:12.191948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:12.257887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d3b24-2b13-40a1-99c6-6812c82a0fa6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:12.381615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b879b3-5a89-4c9b-b2b3-4c8395939b1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:12.382483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b879b3-5a89-4c9b-b2b3-4c8395939b1d service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:12.382772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b879b3-5a89-4c9b-b2b3-4c8395939b1d service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:12.383050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b879b3-5a89-4c9b-b2b3-4c8395939b1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:12.384777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-74b879b3-5a89-4c9b-b2b3-4c8395939b1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:12.447285 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:12.502712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b879b3-5a89-4c9b-b2b3-4c8395939b1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:12.505854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b879b3-5a89-4c9b-b2b3-4c8395939b1d service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:12.534850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 695/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:15.033591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a60bd-36f9-4c56-965c-bda7439a9a8d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:15.034137 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a60bd-36f9-4c56-965c-bda7439a9a8d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:15.034357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a60bd-36f9-4c56-965c-bda7439a9a8d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:15.036329 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 691/1386] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:23:15.041755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124f0ca-ef5d-430d-bc8c-757240ae1bd6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:15.042212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124f0ca-ef5d-430d-bc8c-757240ae1bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:15.042357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124f0ca-ef5d-430d-bc8c-757240ae1bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:15.110904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6124f0ca-ef5d-430d-bc8c-757240ae1bd6 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:15.112122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6124f0ca-ef5d-430d-bc8c-757240ae1bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:15.218624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 696/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:23:15.264811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a60bd-36f9-4c56-965c-bda7439a9a8d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:15.267661 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e46a60bd-36f9-4c56-965c-bda7439a9a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:15.353171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:15.434518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a60bd-36f9-4c56-965c-bda7439a9a8d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:15.593535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b04ac3-571e-4ca9-a05b-4efa3187eca6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:15.594636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04ac3-571e-4ca9-a05b-4efa3187eca6 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:15.595436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04ac3-571e-4ca9-a05b-4efa3187eca6 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:15.596335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04ac3-571e-4ca9-a05b-4efa3187eca6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:15.601506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-26b04ac3-571e-4ca9-a05b-4efa3187eca6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:15.648083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:15.705633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b04ac3-571e-4ca9-a05b-4efa3187eca6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:15.708063 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b04ac3-571e-4ca9-a05b-4efa3187eca6 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:15.726464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 692/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:17.815140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221622f-35e0-4a02-9411-8cc007dc2c74 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:17.815739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221622f-35e0-4a02-9411-8cc007dc2c74 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:17.815739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221622f-35e0-4a02-9411-8cc007dc2c74 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:17.817800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 697/1389] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:17.822922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd1ef-55c4-4be3-a163-5e78094d8215 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:17.823368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd1ef-55c4-4be3-a163-5e78094d8215 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:17.823476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd1ef-55c4-4be3-a163-5e78094d8215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:17.888420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-dbbfd1ef-55c4-4be3-a163-5e78094d8215 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:17.889616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dbbfd1ef-55c4-4be3-a163-5e78094d8215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:17.986460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 693/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:18.033297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221622f-35e0-4a02-9411-8cc007dc2c74 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:18.035014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6221622f-35e0-4a02-9411-8cc007dc2c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:18.070588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:18.186994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221622f-35e0-4a02-9411-8cc007dc2c74 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:18.290218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:18.303094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31bc83-9183-477b-8b7c-fef2e885ec1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:18.303742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31bc83-9183-477b-8b7c-fef2e885ec1e service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:18.304024 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31bc83-9183-477b-8b7c-fef2e885ec1e service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:18.304299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31bc83-9183-477b-8b7c-fef2e885ec1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:18.305911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cf31bc83-9183-477b-8b7c-fef2e885ec1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:18.396629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31bc83-9183-477b-8b7c-fef2e885ec1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:18.398158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31bc83-9183-477b-8b7c-fef2e885ec1e service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:18.416583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 698/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:18.710967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9af3b-1457-4928-9a47-5889f84c474d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:18.711181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9af3b-1457-4928-9a47-5889f84c474d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:18.711464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9af3b-1457-4928-9a47-5889f84c474d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:18.712862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 694/1392] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:18.718230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ff00-a764-48cc-afcc-0e6e265e8a03 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:18.718588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ff00-a764-48cc-afcc-0e6e265e8a03 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:18.718954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ff00-a764-48cc-afcc-0e6e265e8a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:18.785982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7853ff00-a764-48cc-afcc-0e6e265e8a03 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:18.787180 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7853ff00-a764-48cc-afcc-0e6e265e8a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:18.847613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:18.888295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 699/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:18.945233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56d599-632c-41de-82a5-ef7612f8453d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:18.945726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56d599-632c-41de-82a5-ef7612f8453d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:18.945825 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56d599-632c-41de-82a5-ef7612f8453d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:18.947188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 695/1394] 10.222.0.42 () {54 vars in 966 bytes} [Thu Jul 21 06:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:23:18.952295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5d134-c74c-47f5-93d7-d6de880990ad None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:18.952597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5d134-c74c-47f5-93d7-d6de880990ad None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:18.952878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5d134-c74c-47f5-93d7-d6de880990ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:19.030188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1bf5d134-c74c-47f5-93d7-d6de880990ad None None] MFA Rules not processed for user `fa2c8be3105a415ead65579dc18ba40d`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:19.031373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1bf5d134-c74c-47f5-93d7-d6de880990ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:19.111909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 700/1395] 10.222.0.42 () {58 vars in 1063 bytes} [Thu Jul 21 06:23:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:19.120524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56d599-632c-41de-82a5-ef7612f8453d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:19.122073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2b56d599-632c-41de-82a5-ef7612f8453d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:19.136201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:19.228818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56d599-632c-41de-82a5-ef7612f8453d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:19.303090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10844b25-cf12-4c31-a272-724496ce30e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa2c8be3105a415ead65579dc18ba40d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:19.303672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10844b25-cf12-4c31-a272-724496ce30e6 service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:19.303957 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10844b25-cf12-4c31-a272-724496ce30e6 service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:19.304233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10844b25-cf12-4c31-a272-724496ce30e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:19.306030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-10844b25-cf12-4c31-a272-724496ce30e6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:19.373556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:19.417163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10844b25-cf12-4c31-a272-724496ce30e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:19.418654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10844b25-cf12-4c31-a272-724496ce30e6 service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:19.438899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 696/1396] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:20.157879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805962e-83ab-4863-8474-6fa37e7c39cb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:20.158363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805962e-83ab-4863-8474-6fa37e7c39cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:20.158461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805962e-83ab-4863-8474-6fa37e7c39cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:20.230534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2805962e-83ab-4863-8474-6fa37e7c39cb None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:20.231995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2805962e-83ab-4863-8474-6fa37e7c39cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:20.240857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:20.319619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 701/1397] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:23:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:20.362734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c769e1-8616-4e21-88bd-c4f701787301 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:20.363200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c769e1-8616-4e21-88bd-c4f701787301 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:20.363648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c769e1-8616-4e21-88bd-c4f701787301 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:20.364722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 697/1398] 10.222.0.42 () {54 vars in 943 bytes} [Thu Jul 21 06:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:20.371380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac3c88-8594-4af5-b26d-e22d27cd9fb1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:20.371674 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac3c88-8594-4af5-b26d-e22d27cd9fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:20.371955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac3c88-8594-4af5-b26d-e22d27cd9fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:20.446811 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bcac3c88-8594-4af5-b26d-e22d27cd9fb1 None None] MFA Rules not processed for user `94c8eb61f2ca48288cba81d280a30747`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:20.447937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bcac3c88-8594-4af5-b26d-e22d27cd9fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:20.521045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f6ed1-20db-429f-a728-76775f6f0412 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:20.521354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f6ed1-20db-429f-a728-76775f6f0412 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:20.521664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f6ed1-20db-429f-a728-76775f6f0412 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:20.523093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 698/1399] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:23:20.528057 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95276fe4-b7a7-4d75-8cce-42c22d31df8a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:20.528351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95276fe4-b7a7-4d75-8cce-42c22d31df8a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:20.528634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95276fe4-b7a7-4d75-8cce-42c22d31df8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:20.532967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:20.556878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 702/1400] 10.222.0.42 () {58 vars in 1040 bytes} [Thu Jul 21 06:23:20 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:20.566245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac3c88-8594-4af5-b26d-e22d27cd9fb1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:20.576266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac3c88-8594-4af5-b26d-e22d27cd9fb1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:20.596083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-95276fe4-b7a7-4d75-8cce-42c22d31df8a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:20.597579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-95276fe4-b7a7-4d75-8cce-42c22d31df8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:20.682775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db03ce-5ecd-4696-8c86-9e1b549091c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:20.683429 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db03ce-5ecd-4696-8c86-9e1b549091c0 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:20.683708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db03ce-5ecd-4696-8c86-9e1b549091c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:20.683983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db03ce-5ecd-4696-8c86-9e1b549091c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:20.685677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-86db03ce-5ecd-4696-8c86-9e1b549091c0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:20.694777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 699/1401] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:23:20.741248 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95276fe4-b7a7-4d75-8cce-42c22d31df8a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:20.742495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-95276fe4-b7a7-4d75-8cce-42c22d31df8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:20.771760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:20.775743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db03ce-5ecd-4696-8c86-9e1b549091c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:20.777372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db03ce-5ecd-4696-8c86-9e1b549091c0 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:20.785554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 703/1402] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:23:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 06:23:21.005722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95276fe4-b7a7-4d75-8cce-42c22d31df8a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:21.135727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015dc23-6f27-4273-b4e5-5153488d59cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:21.136389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015dc23-6f27-4273-b4e5-5153488d59cc service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:21.136672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015dc23-6f27-4273-b4e5-5153488d59cc service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:21.136950 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015dc23-6f27-4273-b4e5-5153488d59cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:21.138776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2015dc23-6f27-4273-b4e5-5153488d59cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:21.159606 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:21.250449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015dc23-6f27-4273-b4e5-5153488d59cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:21.252051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015dc23-6f27-4273-b4e5-5153488d59cc service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:21.271578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 700/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:23:23.364972 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b323ed-84f1-4cdc-9ce3-7251b6aeddde None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:23.365501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b323ed-84f1-4cdc-9ce3-7251b6aeddde None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:23.365501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b323ed-84f1-4cdc-9ce3-7251b6aeddde None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:23.367298 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 704/1404] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:23.372656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825d4f-2ff0-4e2f-818e-74a576a46d5f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:23.373113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825d4f-2ff0-4e2f-818e-74a576a46d5f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:23.373221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825d4f-2ff0-4e2f-818e-74a576a46d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:23.441854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d2825d4f-2ff0-4e2f-818e-74a576a46d5f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:23.443225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d2825d4f-2ff0-4e2f-818e-74a576a46d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:23.546250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:23.562234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 701/1405] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:23.602191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b323ed-84f1-4cdc-9ce3-7251b6aeddde None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:23.603978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-08b323ed-84f1-4cdc-9ce3-7251b6aeddde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:23.615490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:23.778199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b323ed-84f1-4cdc-9ce3-7251b6aeddde None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:23.874960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:23.913487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ab07a-c013-4a29-a548-081510b2d6d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:23.914186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ab07a-c013-4a29-a548-081510b2d6d5 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:23.914481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ab07a-c013-4a29-a548-081510b2d6d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:23.914762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ab07a-c013-4a29-a548-081510b2d6d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:23.916420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3e4ab07a-c013-4a29-a548-081510b2d6d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:24.020313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ab07a-c013-4a29-a548-081510b2d6d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:24.021926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ab07a-c013-4a29-a548-081510b2d6d5 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:24.042947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 705/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:26.185071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cbd85-6a1d-4330-9019-a713f3c48436 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:26.185670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cbd85-6a1d-4330-9019-a713f3c48436 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:26.185670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cbd85-6a1d-4330-9019-a713f3c48436 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:26.187053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 702/1407] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:23:26.193064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadeb19-7672-4188-a19e-5f0ad1b37dca None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:26.193497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadeb19-7672-4188-a19e-5f0ad1b37dca None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:26.193628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadeb19-7672-4188-a19e-5f0ad1b37dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:26.271733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ccadeb19-7672-4188-a19e-5f0ad1b37dca None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:26.272878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ccadeb19-7672-4188-a19e-5f0ad1b37dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:26.328979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:26.391109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 706/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:26.438534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cbd85-6a1d-4330-9019-a713f3c48436 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:26.440471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-017cbd85-6a1d-4330-9019-a713f3c48436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:26.600751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cbd85-6a1d-4330-9019-a713f3c48436 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:26.675282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:26.725225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2907346-5fcd-4c43-96ab-ea8f952741b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:26.726020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2907346-5fcd-4c43-96ab-ea8f952741b5 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:26.726376 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2907346-5fcd-4c43-96ab-ea8f952741b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:26.726736 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2907346-5fcd-4c43-96ab-ea8f952741b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:26.728439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f2907346-5fcd-4c43-96ab-ea8f952741b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:26.825803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2907346-5fcd-4c43-96ab-ea8f952741b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:26.828397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2907346-5fcd-4c43-96ab-ea8f952741b5 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:26.851759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 703/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:36.391328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=23 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:36.849123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=64 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:43.081443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d14f-f688-4174-bdd4-528aba3178fd None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:43.082042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d14f-f688-4174-bdd4-528aba3178fd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:43.082042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d14f-f688-4174-bdd4-528aba3178fd None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:43.084702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 707/1410] 10.222.0.42 () {54 vars in 918 bytes} [Thu Jul 21 06:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:43.089950 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfeea5-7db6-487b-98f6-e53447f7e17d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:43.090379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfeea5-7db6-487b-98f6-e53447f7e17d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:43.090523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfeea5-7db6-487b-98f6-e53447f7e17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:43.164448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bfdfeea5-7db6-487b-98f6-e53447f7e17d None None] MFA Rules not processed for user `1da2d85eb18049f196aeefc51d3f1e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:43.165641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bfdfeea5-7db6-487b-98f6-e53447f7e17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:43.272331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 704/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Jul 21 06:23:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:43.287052 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0200b-e394-4ea1-877a-6a84f1bf4c4c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:43.287342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0200b-e394-4ea1-877a-6a84f1bf4c4c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:43.287619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0200b-e394-4ea1-877a-6a84f1bf4c4c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:43.288965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 708/1412] 10.222.0.42 () {54 vars in 918 bytes} [Thu Jul 21 06:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:43.293714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f775983-b166-48a9-9dad-50c9badd8084 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:43.294039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f775983-b166-48a9-9dad-50c9badd8084 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:43.294324 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f775983-b166-48a9-9dad-50c9badd8084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:43.364748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:43.367318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6f775983-b166-48a9-9dad-50c9badd8084 None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:43.369062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6f775983-b166-48a9-9dad-50c9badd8084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:43.483898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 705/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Jul 21 06:23:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:46.045691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ad776-ca59-4d46-8293-692af48dd413 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:46.046219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ad776-ca59-4d46-8293-692af48dd413 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:46.046342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ad776-ca59-4d46-8293-692af48dd413 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:46.047617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:46.052957 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c5c66-e5e4-4d47-8650-a9378e29ec74 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:46.053416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c5c66-e5e4-4d47-8650-a9378e29ec74 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:46.053557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c5c66-e5e4-4d47-8650-a9378e29ec74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:46.119455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-af1c5c66-e5e4-4d47-8650-a9378e29ec74 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:46.120639 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-af1c5c66-e5e4-4d47-8650-a9378e29ec74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:46.175555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:46.218192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:23:46.261342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ad776-ca59-4d46-8293-692af48dd413 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:46.262837 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eb1ad776-ca59-4d46-8293-692af48dd413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:46.406838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ad776-ca59-4d46-8293-692af48dd413 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:46.479404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:46.519368 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d747d-39d9-4832-a5e9-5db193caad78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:46.520788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d747d-39d9-4832-a5e9-5db193caad78 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:46.521069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d747d-39d9-4832-a5e9-5db193caad78 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:46.521347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d747d-39d9-4832-a5e9-5db193caad78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:46.523022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2b7d747d-39d9-4832-a5e9-5db193caad78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:46.612846 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d747d-39d9-4832-a5e9-5db193caad78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:46.614537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d747d-39d9-4832-a5e9-5db193caad78 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:46.632510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 710/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:47.530811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109d43d-75ac-4253-8f64-22cb2485b763 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:47.531385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109d43d-75ac-4253-8f64-22cb2485b763 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:47.531385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109d43d-75ac-4253-8f64-22cb2485b763 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:47.532742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:47.537630 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89226001-a84e-4c3b-acad-2ba50f2d7eca None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:47.538084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89226001-a84e-4c3b-acad-2ba50f2d7eca None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:47.538220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89226001-a84e-4c3b-acad-2ba50f2d7eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:47.614541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-89226001-a84e-4c3b-acad-2ba50f2d7eca None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:47.615731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-89226001-a84e-4c3b-acad-2ba50f2d7eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:47.653803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:47.723872 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:47.872357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109d43d-75ac-4253-8f64-22cb2485b763 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:47.873862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b109d43d-75ac-4253-8f64-22cb2485b763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:48.022882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109d43d-75ac-4253-8f64-22cb2485b763 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:48.062886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:48.136820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec575964-079d-49e5-92b3-b23710c0ccf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:48.137478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec575964-079d-49e5-92b3-b23710c0ccf9 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:48.137759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec575964-079d-49e5-92b3-b23710c0ccf9 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:48.138066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec575964-079d-49e5-92b3-b23710c0ccf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:48.139699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ec575964-079d-49e5-92b3-b23710c0ccf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:48.228813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec575964-079d-49e5-92b3-b23710c0ccf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:48.230444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec575964-079d-49e5-92b3-b23710c0ccf9 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:48.248531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 708/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:49.138865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae8fb7-81a0-4bcb-95ae-d3c8d53e86c8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:49.139424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae8fb7-81a0-4bcb-95ae-d3c8d53e86c8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:49.139424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae8fb7-81a0-4bcb-95ae-d3c8d53e86c8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:49.140795 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:49.146458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10be9-b597-4b6d-b52a-ea0d4b995370 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:49.146859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10be9-b597-4b6d-b52a-ea0d4b995370 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:49.147015 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10be9-b597-4b6d-b52a-ea0d4b995370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:49.201708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:49.224491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1ee10be9-b597-4b6d-b52a-ea0d4b995370 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:49.225672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1ee10be9-b597-4b6d-b52a-ea0d4b995370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:49.320999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:49.478446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae8fb7-81a0-4bcb-95ae-d3c8d53e86c8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:49.480031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fcae8fb7-81a0-4bcb-95ae-d3c8d53e86c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:49.626490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae8fb7-81a0-4bcb-95ae-d3c8d53e86c8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:49.641068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:49.743819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc358c5f-dec7-43bb-94f7-57637d2d9bab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:49.744467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc358c5f-dec7-43bb-94f7-57637d2d9bab service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:49.744748 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc358c5f-dec7-43bb-94f7-57637d2d9bab service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:49.745026 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc358c5f-dec7-43bb-94f7-57637d2d9bab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:49.746692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cc358c5f-dec7-43bb-94f7-57637d2d9bab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:49.836496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc358c5f-dec7-43bb-94f7-57637d2d9bab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:49.838127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc358c5f-dec7-43bb-94f7-57637d2d9bab service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:49.855946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 713/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:50.722141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299018a-e126-4419-9afc-9c57775223d7 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:50.722708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299018a-e126-4419-9afc-9c57775223d7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:50.722708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299018a-e126-4419-9afc-9c57775223d7 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:50.724591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:50.729468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223fb4c-71b8-4739-a948-86dfb2c83336 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:50.729922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223fb4c-71b8-4739-a948-86dfb2c83336 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:50.730062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223fb4c-71b8-4739-a948-86dfb2c83336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:50.749189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:50.807721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7223fb4c-71b8-4739-a948-86dfb2c83336 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:50.808919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7223fb4c-71b8-4739-a948-86dfb2c83336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:50.904827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:51.062971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299018a-e126-4419-9afc-9c57775223d7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:51.064504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4299018a-e126-4419-9afc-9c57775223d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:51.174956 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:51.213398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299018a-e126-4419-9afc-9c57775223d7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:51.323696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfe8fa-efc1-40c2-b873-74bb3a96b39e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:51.324252 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfe8fa-efc1-40c2-b873-74bb3a96b39e service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:51.324533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfe8fa-efc1-40c2-b873-74bb3a96b39e service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:51.324808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfe8fa-efc1-40c2-b873-74bb3a96b39e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:51.326446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-74bfe8fa-efc1-40c2-b873-74bb3a96b39e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:51.399972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:51.414084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfe8fa-efc1-40c2-b873-74bb3a96b39e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:51.415657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfe8fa-efc1-40c2-b873-74bb3a96b39e service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:51.433575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 711/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:52.294574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a82ee-614d-40e2-bb9c-2832996979f1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:52.295151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a82ee-614d-40e2-bb9c-2832996979f1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:52.295151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a82ee-614d-40e2-bb9c-2832996979f1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:52.296501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:52.302279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8ebf5-d41f-4524-a5ac-5afd3c83beca None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:52.302706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8ebf5-d41f-4524-a5ac-5afd3c83beca None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:52.302815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8ebf5-d41f-4524-a5ac-5afd3c83beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:52.375608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-77a8ebf5-d41f-4524-a5ac-5afd3c83beca None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:52.376726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-77a8ebf5-d41f-4524-a5ac-5afd3c83beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:52.471125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:23:52.620641 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a82ee-614d-40e2-bb9c-2832996979f1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:52.622191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4f7a82ee-614d-40e2-bb9c-2832996979f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:52.706081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:52.767721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a82ee-614d-40e2-bb9c-2832996979f1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:52.881058 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50927a23-2002-4325-9896-aa7041490ca7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:52.881710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50927a23-2002-4325-9896-aa7041490ca7 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:52.882025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50927a23-2002-4325-9896-aa7041490ca7 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:52.882308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50927a23-2002-4325-9896-aa7041490ca7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:52.883945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-50927a23-2002-4325-9896-aa7041490ca7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:52.931932 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:52.973539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50927a23-2002-4325-9896-aa7041490ca7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:52.975256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50927a23-2002-4325-9896-aa7041490ca7 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:52.993337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 716/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:53.810995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc2bde-f18a-4c3a-96cf-036e39c02912 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:53.811591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc2bde-f18a-4c3a-96cf-036e39c02912 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:53.811591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc2bde-f18a-4c3a-96cf-036e39c02912 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:53.812934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:53.818292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84a847-399e-40b1-96f6-c7952a1e7d40 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:53.818739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84a847-399e-40b1-96f6-c7952a1e7d40 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:53.818847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84a847-399e-40b1-96f6-c7952a1e7d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:53.893120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0f84a847-399e-40b1-96f6-c7952a1e7d40 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:53.894333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0f84a847-399e-40b1-96f6-c7952a1e7d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:53.991140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:54.142999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc2bde-f18a-4c3a-96cf-036e39c02912 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:54.144442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f0fc2bde-f18a-4c3a-96cf-036e39c02912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:54.194097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:54.295922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc2bde-f18a-4c3a-96cf-036e39c02912 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:54.408727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b60014-c000-47d0-8055-850f72837237 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:54.409306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b60014-c000-47d0-8055-850f72837237 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:54.409592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b60014-c000-47d0-8055-850f72837237 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:54.409873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b60014-c000-47d0-8055-850f72837237 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:54.411555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e6b60014-c000-47d0-8055-850f72837237 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:54.418501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:54.500451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b60014-c000-47d0-8055-850f72837237 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:54.502073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b60014-c000-47d0-8055-850f72837237 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:54.521079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 714/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:55.319191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7708362-d9da-476d-b8fe-b1ee3dc65d6d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:55.319731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7708362-d9da-476d-b8fe-b1ee3dc65d6d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:55.319731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7708362-d9da-476d-b8fe-b1ee3dc65d6d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:55.321052 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:55.325877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864af13-ff4e-40c6-a8f1-2ab99293fb41 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:55.326335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864af13-ff4e-40c6-a8f1-2ab99293fb41 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:55.326475 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864af13-ff4e-40c6-a8f1-2ab99293fb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:55.399441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8864af13-ff4e-40c6-a8f1-2ab99293fb41 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:55.400643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8864af13-ff4e-40c6-a8f1-2ab99293fb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:55.468261 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:55.497897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:55.648625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7708362-d9da-476d-b8fe-b1ee3dc65d6d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:55.650096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e7708362-d9da-476d-b8fe-b1ee3dc65d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:55.667931 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:55.796090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7708362-d9da-476d-b8fe-b1ee3dc65d6d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:55.883968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:55.910869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420901bb-0716-4b1a-b8f0-abf42638ae5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:55.911432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420901bb-0716-4b1a-b8f0-abf42638ae5c service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:55.911713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420901bb-0716-4b1a-b8f0-abf42638ae5c service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:55.911992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420901bb-0716-4b1a-b8f0-abf42638ae5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:55.913627 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-420901bb-0716-4b1a-b8f0-abf42638ae5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:56.002240 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420901bb-0716-4b1a-b8f0-abf42638ae5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:56.003836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420901bb-0716-4b1a-b8f0-abf42638ae5c service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:56.024844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 719/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:56.832596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756b811-0790-45c7-82f5-1d075cfa17d9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:56.833591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756b811-0790-45c7-82f5-1d075cfa17d9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:56.833591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756b811-0790-45c7-82f5-1d075cfa17d9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:56.834703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:56.839660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5066-bdb4-4fe9-bf29-3f6a3fbb5236 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:56.840092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5066-bdb4-4fe9-bf29-3f6a3fbb5236 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:56.840209 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5066-bdb4-4fe9-bf29-3f6a3fbb5236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:56.916166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-239f5066-bdb4-4fe9-bf29-3f6a3fbb5236 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:56.917378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-239f5066-bdb4-4fe9-bf29-3f6a3fbb5236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:56.959903 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:57.015621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:57.170693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756b811-0790-45c7-82f5-1d075cfa17d9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:57.172158 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5756b811-0790-45c7-82f5-1d075cfa17d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:57.320167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756b811-0790-45c7-82f5-1d075cfa17d9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:57.373624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:57.433059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab6a61-1afc-4361-85fc-184776b4c55c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:57.433634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab6a61-1afc-4361-85fc-184776b4c55c service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:57.433915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab6a61-1afc-4361-85fc-184776b4c55c service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:57.434216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab6a61-1afc-4361-85fc-184776b4c55c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:57.435832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-abab6a61-1afc-4361-85fc-184776b4c55c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:57.525501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab6a61-1afc-4361-85fc-184776b4c55c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:57.527110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab6a61-1afc-4361-85fc-184776b4c55c service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:57.544844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 717/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:23:58.362161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5b180-fc1b-43d2-b759-e5b78b93e580 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:58.362821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5b180-fc1b-43d2-b759-e5b78b93e580 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:58.362821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5b180-fc1b-43d2-b759-e5b78b93e580 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:58.364898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:23:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:23:58.369768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1037-955e-4f55-9c85-14d34d4ceabf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:58.370252 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1037-955e-4f55-9c85-14d34d4ceabf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:58.370365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1037-955e-4f55-9c85-14d34d4ceabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:58.445377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-55fe1037-955e-4f55-9c85-14d34d4ceabf None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:23:58.446592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-55fe1037-955e-4f55-9c85-14d34d4ceabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:58.448578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:58.547542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:23:58.699145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5b180-fc1b-43d2-b759-e5b78b93e580 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:23:58.700585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e8c5b180-fc1b-43d2-b759-e5b78b93e580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:58.845951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5b180-fc1b-43d2-b759-e5b78b93e580 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:23:58.873542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:23:58.959279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7175c12-e784-401d-a6d5-f195686749a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:23:58.959852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7175c12-e784-401d-a6d5-f195686749a8 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:23:58.960132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7175c12-e784-401d-a6d5-f195686749a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:23:58.960404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7175c12-e784-401d-a6d5-f195686749a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:23:58.962051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f7175c12-e784-401d-a6d5-f195686749a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:23:59.050533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7175c12-e784-401d-a6d5-f195686749a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:23:59.052693 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7175c12-e784-401d-a6d5-f195686749a8 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:23:59.070863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 722/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:23:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:00.990336 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22baba-9773-4a29-a20c-fd66fd1a8f0d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:00.990909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22baba-9773-4a29-a20c-fd66fd1a8f0d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:00.990909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22baba-9773-4a29-a20c-fd66fd1a8f0d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:00.992284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:00.997666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974778-b580-4546-9fe5-5dc184c3376e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:00.998136 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974778-b580-4546-9fe5-5dc184c3376e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:00.998307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974778-b580-4546-9fe5-5dc184c3376e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:01.038775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:01.068881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-29974778-b580-4546-9fe5-5dc184c3376e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:01.070130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-29974778-b580-4546-9fe5-5dc184c3376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:01.170365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:01.210115 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22baba-9773-4a29-a20c-fd66fd1a8f0d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:01.211643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ea22baba-9773-4a29-a20c-fd66fd1a8f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:01.313810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22baba-9773-4a29-a20c-fd66fd1a8f0d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:01.345851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:01.387311 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9db761-ef28-4967-a8a9-b7bbe05f1427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:01.387961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9db761-ef28-4967-a8a9-b7bbe05f1427 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:01.388246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9db761-ef28-4967-a8a9-b7bbe05f1427 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:01.388521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9db761-ef28-4967-a8a9-b7bbe05f1427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:01.390240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6d9db761-ef28-4967-a8a9-b7bbe05f1427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:01.496657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9db761-ef28-4967-a8a9-b7bbe05f1427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:01.498107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9db761-ef28-4967-a8a9-b7bbe05f1427 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:01.516724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 720/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:03.727803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12253-99ba-4ef5-92dc-5526a1dea6e2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:03.728643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12253-99ba-4ef5-92dc-5526a1dea6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:03.728804 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12253-99ba-4ef5-92dc-5526a1dea6e2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:03.730321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:24:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:03.736052 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637b00e-97b5-460e-b4e1-2b907d95767e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:03.736533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637b00e-97b5-460e-b4e1-2b907d95767e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:03.736639 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637b00e-97b5-460e-b4e1-2b907d95767e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:03.806361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b637b00e-97b5-460e-b4e1-2b907d95767e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:03.807587 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b637b00e-97b5-460e-b4e1-2b907d95767e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:03.821297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:03.920972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:03.970534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa12253-99ba-4ef5-92dc-5526a1dea6e2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:03.972081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8aa12253-99ba-4ef5-92dc-5526a1dea6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:04.090303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa12253-99ba-4ef5-92dc-5526a1dea6e2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:04.102650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:04.172095 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96733d0-4ea6-4eec-9540-f3793d17989c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:04.172746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96733d0-4ea6-4eec-9540-f3793d17989c service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:04.173036 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96733d0-4ea6-4eec-9540-f3793d17989c service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:04.173326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96733d0-4ea6-4eec-9540-f3793d17989c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:04.175124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d96733d0-4ea6-4eec-9540-f3793d17989c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:04.307543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96733d0-4ea6-4eec-9540-f3793d17989c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:04.308883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96733d0-4ea6-4eec-9540-f3793d17989c service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:04.328228 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 725/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:06.654230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eabce3-f1c7-4399-a913-610c3be90e18 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:06.654804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eabce3-f1c7-4399-a913-610c3be90e18 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:06.654804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eabce3-f1c7-4399-a913-610c3be90e18 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:06.656169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:06.662206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef2b34-c2e1-4bec-bf09-056ff8beefd4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:06.662599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef2b34-c2e1-4bec-bf09-056ff8beefd4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:06.662766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef2b34-c2e1-4bec-bf09-056ff8beefd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:06.710610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:06.729902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-56ef2b34-c2e1-4bec-bf09-056ff8beefd4 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:06.731120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-56ef2b34-c2e1-4bec-bf09-056ff8beefd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:06.825085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:06.864368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eabce3-f1c7-4399-a913-610c3be90e18 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:06.865902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a0eabce3-f1c7-4399-a913-610c3be90e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:06.967224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eabce3-f1c7-4399-a913-610c3be90e18 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:07.007318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:07.042084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcffe1-3184-4f75-8705-0e88eba318ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:07.042740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcffe1-3184-4f75-8705-0e88eba318ae service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:07.043024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcffe1-3184-4f75-8705-0e88eba318ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:07.043300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcffe1-3184-4f75-8705-0e88eba318ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:07.044949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a1dcffe1-3184-4f75-8705-0e88eba318ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:07.154697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcffe1-3184-4f75-8705-0e88eba318ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:07.156138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcffe1-3184-4f75-8705-0e88eba318ae service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:07.173977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 723/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:09.328487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf6e35-bb88-42ca-87a0-a07c07eb4435 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:09.328996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf6e35-bb88-42ca-87a0-a07c07eb4435 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:09.328996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf6e35-bb88-42ca-87a0-a07c07eb4435 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:09.330362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:09.336351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2ad02-0606-4606-8836-77766636cf8e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:09.336777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2ad02-0606-4606-8836-77766636cf8e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:09.336956 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2ad02-0606-4606-8836-77766636cf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:09.402008 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6fc2ad02-0606-4606-8836-77766636cf8e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:09.403219 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6fc2ad02-0606-4606-8836-77766636cf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:09.423344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:09.501192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:09.540948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf6e35-bb88-42ca-87a0-a07c07eb4435 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:09.542504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-75bf6e35-bb88-42ca-87a0-a07c07eb4435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:09.648341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf6e35-bb88-42ca-87a0-a07c07eb4435 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:09.668443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:09.722845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13adcf-1a58-4e28-a1bd-5d241c26789c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:09.724291 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13adcf-1a58-4e28-a1bd-5d241c26789c service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:09.724677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13adcf-1a58-4e28-a1bd-5d241c26789c service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:09.724965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13adcf-1a58-4e28-a1bd-5d241c26789c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:09.726982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3d13adcf-1a58-4e28-a1bd-5d241c26789c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:09.837968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13adcf-1a58-4e28-a1bd-5d241c26789c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:09.839411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13adcf-1a58-4e28-a1bd-5d241c26789c service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:09.857571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 728/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:12.016972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228cab1-5aee-45c4-83e2-7f711745ddff None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:12.017512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228cab1-5aee-45c4-83e2-7f711745ddff None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:12.017512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228cab1-5aee-45c4-83e2-7f711745ddff None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:12.018939 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:12.025198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b466788-b647-4f21-aea0-20a46bb90456 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:12.025612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b466788-b647-4f21-aea0-20a46bb90456 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:12.025753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b466788-b647-4f21-aea0-20a46bb90456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:12.088552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:12.091328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4b466788-b647-4f21-aea0-20a46bb90456 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:12.093137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4b466788-b647-4f21-aea0-20a46bb90456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:12.189617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:12.229284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228cab1-5aee-45c4-83e2-7f711745ddff None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:12.230897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0228cab1-5aee-45c4-83e2-7f711745ddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:12.333934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228cab1-5aee-45c4-83e2-7f711745ddff None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:12.381555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:12.406108 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b69cb-58af-4736-9b41-94b913d86097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:12.406756 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b69cb-58af-4736-9b41-94b913d86097 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:12.407044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b69cb-58af-4736-9b41-94b913d86097 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:12.407326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b69cb-58af-4736-9b41-94b913d86097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:12.408958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e28b69cb-58af-4736-9b41-94b913d86097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:12.516941 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b69cb-58af-4736-9b41-94b913d86097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:12.518403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b69cb-58af-4736-9b41-94b913d86097 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:12.536771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 726/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:14.635986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f134ee-0d0f-4a82-b10f-08663e09b737 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:14.636527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f134ee-0d0f-4a82-b10f-08663e09b737 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:14.636527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f134ee-0d0f-4a82-b10f-08663e09b737 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:14.637895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:14.643170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813bec-6dc7-404e-922e-6b8b680d2623 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:14.643605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813bec-6dc7-404e-922e-6b8b680d2623 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:14.643717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813bec-6dc7-404e-922e-6b8b680d2623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:14.706807 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-16813bec-6dc7-404e-922e-6b8b680d2623 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:14.708185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-16813bec-6dc7-404e-922e-6b8b680d2623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:14.736040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:14.803334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:14.843638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f134ee-0d0f-4a82-b10f-08663e09b737 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:14.845251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-84f134ee-0d0f-4a82-b10f-08663e09b737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:14.948491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f134ee-0d0f-4a82-b10f-08663e09b737 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:14.978010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:15.023813 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3638741-cc04-4e97-b4ee-81d18eb0439c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:15.024470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3638741-cc04-4e97-b4ee-81d18eb0439c service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:15.024757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3638741-cc04-4e97-b4ee-81d18eb0439c service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:15.025037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3638741-cc04-4e97-b4ee-81d18eb0439c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:15.026728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e3638741-cc04-4e97-b4ee-81d18eb0439c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:15.138959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3638741-cc04-4e97-b4ee-81d18eb0439c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:15.140412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3638741-cc04-4e97-b4ee-81d18eb0439c service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:15.158557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 731/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:17.255848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597718cf-cbca-41bd-a54b-1fa3714d671e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:17.256401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597718cf-cbca-41bd-a54b-1fa3714d671e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:17.256401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597718cf-cbca-41bd-a54b-1fa3714d671e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:17.257828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:17.263179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06f907-f006-4828-9bf6-8af9ca70073d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:17.263559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06f907-f006-4828-9bf6-8af9ca70073d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:17.263743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06f907-f006-4828-9bf6-8af9ca70073d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:17.328063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fb06f907-f006-4828-9bf6-8af9ca70073d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:17.329184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fb06f907-f006-4828-9bf6-8af9ca70073d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:17.337839 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:17.426447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:17.464708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597718cf-cbca-41bd-a54b-1fa3714d671e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:17.466256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-597718cf-cbca-41bd-a54b-1fa3714d671e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:17.568922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597718cf-cbca-41bd-a54b-1fa3714d671e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:17.623191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:17.643230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29574b58-2ed8-40b2-a8be-99f1780c553e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:17.643877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574b58-2ed8-40b2-a8be-99f1780c553e service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:17.644160 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574b58-2ed8-40b2-a8be-99f1780c553e service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:17.644431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574b58-2ed8-40b2-a8be-99f1780c553e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:17.647434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-29574b58-2ed8-40b2-a8be-99f1780c553e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:17.752547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29574b58-2ed8-40b2-a8be-99f1780c553e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:17.753996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29574b58-2ed8-40b2-a8be-99f1780c553e service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:17.775952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 729/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:19.862753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb23a0-f37e-4af4-8f17-4eb0ca0ba4c0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:19.863295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb23a0-f37e-4af4-8f17-4eb0ca0ba4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:19.863295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb23a0-f37e-4af4-8f17-4eb0ca0ba4c0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:19.865256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:19.870529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2149b-393e-4681-8abc-6c42bfe96c2f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:19.870926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2149b-393e-4681-8abc-6c42bfe96c2f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:19.871096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2149b-393e-4681-8abc-6c42bfe96c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:19.934401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2cb2149b-393e-4681-8abc-6c42bfe96c2f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:19.935588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2cb2149b-393e-4681-8abc-6c42bfe96c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:19.972503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:20.030781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:20.068996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb23a0-f37e-4af4-8f17-4eb0ca0ba4c0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:20.070567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2bbb23a0-f37e-4af4-8f17-4eb0ca0ba4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:20.174321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb23a0-f37e-4af4-8f17-4eb0ca0ba4c0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:20.213737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:20.249591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48275986-c825-4192-a380-34d913800bf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:20.250259 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48275986-c825-4192-a380-34d913800bf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:20.250545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48275986-c825-4192-a380-34d913800bf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:20.250821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48275986-c825-4192-a380-34d913800bf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:20.252454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-48275986-c825-4192-a380-34d913800bf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:20.369507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48275986-c825-4192-a380-34d913800bf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:20.370978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48275986-c825-4192-a380-34d913800bf6 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:20.389152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 734/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:24:22.450843 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d3762-3649-449f-a634-ec17dc7a5944 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:22.451377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d3762-3649-449f-a634-ec17dc7a5944 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:22.451377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d3762-3649-449f-a634-ec17dc7a5944 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:22.452710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:22.458602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d605bf-ddb7-4a49-9c8f-d68d8728593c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:22.459051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d605bf-ddb7-4a49-9c8f-d68d8728593c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:22.459380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d605bf-ddb7-4a49-9c8f-d68d8728593c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:22.524319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-28d605bf-ddb7-4a49-9c8f-d68d8728593c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:22.525510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-28d605bf-ddb7-4a49-9c8f-d68d8728593c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:22.543731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:22.621882 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:22.659220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d3762-3649-449f-a634-ec17dc7a5944 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:22.660812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7a9d3762-3649-449f-a634-ec17dc7a5944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:22.765486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d3762-3649-449f-a634-ec17dc7a5944 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:22.831168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:22.838769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0c6f4-96b0-4862-ac82-3f6ca0ac7388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:22.839416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0c6f4-96b0-4862-ac82-3f6ca0ac7388 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:22.839700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0c6f4-96b0-4862-ac82-3f6ca0ac7388 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:22.839978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0c6f4-96b0-4862-ac82-3f6ca0ac7388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:22.841611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-06a0c6f4-96b0-4862-ac82-3f6ca0ac7388 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:22.948961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0c6f4-96b0-4862-ac82-3f6ca0ac7388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:22.950450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0c6f4-96b0-4862-ac82-3f6ca0ac7388 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:22.969449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 732/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:25.033662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471dbaf-39cd-4d14-81f2-9d4ed61bd947 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:25.034267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471dbaf-39cd-4d14-81f2-9d4ed61bd947 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:25.034267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471dbaf-39cd-4d14-81f2-9d4ed61bd947 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:25.035606 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:25.044567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0dc5b-af7c-40c6-85d2-56d3847eb333 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:25.044567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0dc5b-af7c-40c6-85d2-56d3847eb333 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:25.044567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0dc5b-af7c-40c6-85d2-56d3847eb333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:25.107723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-70d0dc5b-af7c-40c6-85d2-56d3847eb333 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:25.108916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-70d0dc5b-af7c-40c6-85d2-56d3847eb333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:25.156008 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:25.204810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:25.242839 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471dbaf-39cd-4d14-81f2-9d4ed61bd947 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:25.244379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e471dbaf-39cd-4d14-81f2-9d4ed61bd947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:25.346802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471dbaf-39cd-4d14-81f2-9d4ed61bd947 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:25.393764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:25.421015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45b5c1-343f-439e-a082-feb4b70ff9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:25.421662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45b5c1-343f-439e-a082-feb4b70ff9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:25.421950 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45b5c1-343f-439e-a082-feb4b70ff9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:25.422250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45b5c1-343f-439e-a082-feb4b70ff9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:25.423875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fd45b5c1-343f-439e-a082-feb4b70ff9e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:25.534108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45b5c1-343f-439e-a082-feb4b70ff9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:25.535539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45b5c1-343f-439e-a082-feb4b70ff9e6 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:25.554669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 737/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:35.202941 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=22 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:35.551873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=66 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:45.387543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f83e7f-a6e3-4d01-b355-7f3524da709b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:45.388082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f83e7f-a6e3-4d01-b355-7f3524da709b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:45.388082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f83e7f-a6e3-4d01-b355-7f3524da709b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:45.389398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 734/1471] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:45.394796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0c408-9d9f-4257-80bf-8bdcf1293158 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:45.395252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0c408-9d9f-4257-80bf-8bdcf1293158 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:45.395355 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0c408-9d9f-4257-80bf-8bdcf1293158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:45.461608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0be0c408-9d9f-4257-80bf-8bdcf1293158 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:45.462842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0be0c408-9d9f-4257-80bf-8bdcf1293158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:45.563740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 738/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:45.650218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47592701-6659-4a76-a2e9-8ce0c9ffc6f9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:45.650509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47592701-6659-4a76-a2e9-8ce0c9ffc6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:45.650791 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47592701-6659-4a76-a2e9-8ce0c9ffc6f9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:45.652152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 735/1473] 10.222.0.42 () {54 vars in 966 bytes} [Thu Jul 21 06:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:45.656879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80964eb2-57f2-470f-b247-10614ab8c3e5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:45.657172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80964eb2-57f2-470f-b247-10614ab8c3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:45.657454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80964eb2-57f2-470f-b247-10614ab8c3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:45.732194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-80964eb2-57f2-470f-b247-10614ab8c3e5 None None] MFA Rules not processed for user `fa2c8be3105a415ead65579dc18ba40d`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:45.733383 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-80964eb2-57f2-470f-b247-10614ab8c3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:45.753589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:45.809176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 739/1474] 10.222.0.42 () {58 vars in 1063 bytes} [Thu Jul 21 06:24:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:45.816586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47592701-6659-4a76-a2e9-8ce0c9ffc6f9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:45.818328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-47592701-6659-4a76-a2e9-8ce0c9ffc6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:45.921512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47592701-6659-4a76-a2e9-8ce0c9ffc6f9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:45.993127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e2304-a41c-467c-a7f3-536a0e7816be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa2c8be3105a415ead65579dc18ba40d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:45.993805 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e2304-a41c-467c-a7f3-536a0e7816be service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:45.994125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e2304-a41c-467c-a7f3-536a0e7816be service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:45.994406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e2304-a41c-467c-a7f3-536a0e7816be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:45.996043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3e0e2304-a41c-467c-a7f3-536a0e7816be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:46.053350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:46.109606 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e2304-a41c-467c-a7f3-536a0e7816be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:46.111090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e2304-a41c-467c-a7f3-536a0e7816be service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:46.129479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 736/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:24:48.250384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249579be-3cdf-42e3-b129-399f4b8d3859 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:48.250919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249579be-3cdf-42e3-b129-399f4b8d3859 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:48.250919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249579be-3cdf-42e3-b129-399f4b8d3859 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:48.252310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 740/1476] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:48.257992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de958b7-ec86-4c1b-83b1-ac0777dcdbd1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:48.258418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de958b7-ec86-4c1b-83b1-ac0777dcdbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:48.258580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de958b7-ec86-4c1b-83b1-ac0777dcdbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:48.324280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4de958b7-ec86-4c1b-83b1-ac0777dcdbd1 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:48.325384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4de958b7-ec86-4c1b-83b1-ac0777dcdbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:48.420999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 737/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:48.444920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249579be-3cdf-42e3-b129-399f4b8d3859 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:48.446530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-249579be-3cdf-42e3-b129-399f4b8d3859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:48.571446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249579be-3cdf-42e3-b129-399f4b8d3859 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:48.619915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:48.666856 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91996c48-01c9-48e5-8ee1-dc925e29abb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:48.667510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91996c48-01c9-48e5-8ee1-dc925e29abb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:48.667793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91996c48-01c9-48e5-8ee1-dc925e29abb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:48.668074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91996c48-01c9-48e5-8ee1-dc925e29abb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:48.668803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91996c48-01c9-48e5-8ee1-dc925e29abb7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:48.670587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-91996c48-01c9-48e5-8ee1-dc925e29abb7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:24:48.670587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 21 06:24:48.670587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: Jul 21 06:24:48.671725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91996c48-01c9-48e5-8ee1-dc925e29abb7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:48.690179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 741/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 06:24:48 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:24:50.730521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3e55-ed9f-44cf-bc25-0b47e0293cb0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:50.731084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3e55-ed9f-44cf-bc25-0b47e0293cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:50.731084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3e55-ed9f-44cf-bc25-0b47e0293cb0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:50.732456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 738/1479] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:50.737564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e75e3-e6eb-48a3-8c8e-70e0cc9039ee None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:50.738014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e75e3-e6eb-48a3-8c8e-70e0cc9039ee None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:50.738161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e75e3-e6eb-48a3-8c8e-70e0cc9039ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:50.802903 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fd4e75e3-e6eb-48a3-8c8e-70e0cc9039ee None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:50.804130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fd4e75e3-e6eb-48a3-8c8e-70e0cc9039ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:50.901313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 742/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:50.945144 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a3e55-ed9f-44cf-bc25-0b47e0293cb0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:50.946771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bc6a3e55-ed9f-44cf-bc25-0b47e0293cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:50.956286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:51.094098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a3e55-ed9f-44cf-bc25-0b47e0293cb0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:51.173911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:51.208188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb2084-14e6-49a5-ab7b-368677d63a59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:51.208967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb2084-14e6-49a5-ab7b-368677d63a59 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:51.209260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb2084-14e6-49a5-ab7b-368677d63a59 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:51.209544 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb2084-14e6-49a5-ab7b-368677d63a59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:51.211302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5bbb2084-14e6-49a5-ab7b-368677d63a59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:51.299166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb2084-14e6-49a5-ab7b-368677d63a59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:51.300761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb2084-14e6-49a5-ab7b-368677d63a59 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:51.319604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 739/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:53.550314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7565bf-e55a-44c6-a9ba-eed5064270ee None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:53.550818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7565bf-e55a-44c6-a9ba-eed5064270ee None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:53.550929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7565bf-e55a-44c6-a9ba-eed5064270ee None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:53.552250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 743/1482] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:53.557676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92764faa-b1f7-4cc4-8572-61557cb7e0bf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:53.558141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92764faa-b1f7-4cc4-8572-61557cb7e0bf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:53.558279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92764faa-b1f7-4cc4-8572-61557cb7e0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:53.621616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-92764faa-b1f7-4cc4-8572-61557cb7e0bf None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:53.622845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-92764faa-b1f7-4cc4-8572-61557cb7e0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:53.669456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:53.719751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 740/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:53.756744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7565bf-e55a-44c6-a9ba-eed5064270ee None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:53.758387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1d7565bf-e55a-44c6-a9ba-eed5064270ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:53.783126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:53.862997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7565bf-e55a-44c6-a9ba-eed5064270ee None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:53.936336 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f3c16-b599-4df3-af47-037327772e7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:53.936981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f3c16-b599-4df3-af47-037327772e7d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:53.937267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f3c16-b599-4df3-af47-037327772e7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:53.937547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f3c16-b599-4df3-af47-037327772e7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:53.939231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-608f3c16-b599-4df3-af47-037327772e7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:54.013472 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:54.048496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f3c16-b599-4df3-af47-037327772e7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:54.049979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f3c16-b599-4df3-af47-037327772e7d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:54.071144 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 744/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:56.156813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a46616-a1f6-47ac-b2e6-3cecfa00df39 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:56.157260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a46616-a1f6-47ac-b2e6-3cecfa00df39 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:56.157829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a46616-a1f6-47ac-b2e6-3cecfa00df39 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:56.158799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 741/1485] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:56.164780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784ea74-a5d7-493b-95ba-4c2e917fb5f4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:56.165219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784ea74-a5d7-493b-95ba-4c2e917fb5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:56.165323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784ea74-a5d7-493b-95ba-4c2e917fb5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:56.237886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3784ea74-a5d7-493b-95ba-4c2e917fb5f4 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:56.239223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3784ea74-a5d7-493b-95ba-4c2e917fb5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:56.335115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 745/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:56.373603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a46616-a1f6-47ac-b2e6-3cecfa00df39 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:56.375198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-92a46616-a1f6-47ac-b2e6-3cecfa00df39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:56.479792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a46616-a1f6-47ac-b2e6-3cecfa00df39 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:56.551018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:56.553856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7645df-ff50-4b68-b153-4d0c9ca3955f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:56.554722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7645df-ff50-4b68-b153-4d0c9ca3955f service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:56.555213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7645df-ff50-4b68-b153-4d0c9ca3955f service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:56.555512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7645df-ff50-4b68-b153-4d0c9ca3955f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:56.557433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5a7645df-ff50-4b68-b153-4d0c9ca3955f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:56.665147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7645df-ff50-4b68-b153-4d0c9ca3955f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:56.666538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7645df-ff50-4b68-b153-4d0c9ca3955f service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:56.685396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 742/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:24:58.879324 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977b60b-b68d-4d18-834f-264933c7c74d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:58.879904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977b60b-b68d-4d18-834f-264933c7c74d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:58.879904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977b60b-b68d-4d18-834f-264933c7c74d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:58.881829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 746/1488] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:24:58.887064 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967696ee-c6f1-41cd-b203-e1cde8afd6c2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:58.887510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967696ee-c6f1-41cd-b203-e1cde8afd6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:58.887657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967696ee-c6f1-41cd-b203-e1cde8afd6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:58.951360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-967696ee-c6f1-41cd-b203-e1cde8afd6c2 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:24:58.952562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-967696ee-c6f1-41cd-b203-e1cde8afd6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:59.004344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:59.050462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 743/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:24:59.091553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977b60b-b68d-4d18-834f-264933c7c74d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:24:59.093081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e977b60b-b68d-4d18-834f-264933c7c74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:59.128038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:59.196145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977b60b-b68d-4d18-834f-264933c7c74d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:24:59.269254 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f8e13-2f01-4ad6-aaf8-04e35b17832d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:24:59.269902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f8e13-2f01-4ad6-aaf8-04e35b17832d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:24:59.270222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f8e13-2f01-4ad6-aaf8-04e35b17832d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:24:59.270501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f8e13-2f01-4ad6-aaf8-04e35b17832d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:24:59.272179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b74f8e13-2f01-4ad6-aaf8-04e35b17832d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:24:59.351005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:24:59.380783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f8e13-2f01-4ad6-aaf8-04e35b17832d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:24:59.382852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f8e13-2f01-4ad6-aaf8-04e35b17832d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:24:59.401086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 747/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:24:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:25:01.665249 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9afdc-2c1e-4fb7-afa2-a41415b44fa5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:25:01.665770 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9afdc-2c1e-4fb7-afa2-a41415b44fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:25:01.665770 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9afdc-2c1e-4fb7-afa2-a41415b44fa5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:25:01.667167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 744/1491] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 06:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:25:01.672688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ff5c2-2d58-440e-ab04-7f8696b9591a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:25:01.673172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ff5c2-2d58-440e-ab04-7f8696b9591a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:25:01.673276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ff5c2-2d58-440e-ab04-7f8696b9591a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:25:01.743259 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fd3ff5c2-2d58-440e-ab04-7f8696b9591a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:25:01.744715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fd3ff5c2-2d58-440e-ab04-7f8696b9591a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:25:01.842124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 748/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 06:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:25:01.883389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9afdc-2c1e-4fb7-afa2-a41415b44fa5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:25:01.884918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cbb9afdc-2c1e-4fb7-afa2-a41415b44fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:25:01.989822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9afdc-2c1e-4fb7-afa2-a41415b44fa5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:25:02.065600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b3896-b5f9-41bb-984b-c44b1eeccd7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:25:02.066289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b3896-b5f9-41bb-984b-c44b1eeccd7a service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:25:02.066574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b3896-b5f9-41bb-984b-c44b1eeccd7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:25:02.066852 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b3896-b5f9-41bb-984b-c44b1eeccd7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:25:02.068560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f12b3896-b5f9-41bb-984b-c44b1eeccd7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:25:02.077642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:25:02.176336 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b3896-b5f9-41bb-984b-c44b1eeccd7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:25:02.178719 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b3896-b5f9-41bb-984b-c44b1eeccd7a service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:25:02.196247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 745/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:25:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:25:11.840143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=82 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:25:12.193861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=50 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:35.020358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10d41-849c-4df8-904e-6b95cab1e8e0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:35.020946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10d41-849c-4df8-904e-6b95cab1e8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:35.020946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10d41-849c-4df8-904e-6b95cab1e8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:35.098587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-65f10d41-849c-4df8-904e-6b95cab1e8e0 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:26:35.099790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-65f10d41-849c-4df8-904e-6b95cab1e8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:35.196505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 749/1494] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:26:35.204580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe3df3-50d6-4389-bcfa-4d5bfd127d22 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:35.205060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe3df3-50d6-4389-bcfa-4d5bfd127d22 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:35.205167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe3df3-50d6-4389-bcfa-4d5bfd127d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:35.278799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-32fe3df3-50d6-4389-bcfa-4d5bfd127d22 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:26:35.279918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-32fe3df3-50d6-4389-bcfa-4d5bfd127d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:35.375049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 746/1495] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:26:35.379672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f10d41-849c-4df8-904e-6b95cab1e8e0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:26:35.380881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-65f10d41-849c-4df8-904e-6b95cab1e8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:35.392225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f10d41-849c-4df8-904e-6b95cab1e8e0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:26:35.455952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:35.490102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0a210-73f2-47d5-9ae5-c82a0c9b4f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:26:35.490778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0a210-73f2-47d5-9ae5-c82a0c9b4f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:35.491063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0a210-73f2-47d5-9ae5-c82a0c9b4f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:35.491344 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0a210-73f2-47d5-9ae5-c82a0c9b4f4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:35.492068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0a210-73f2-47d5-9ae5-c82a0c9b4f4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:26:35.492378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0a210-73f2-47d5-9ae5-c82a0c9b4f4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:26:35.493391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-8ab0a210-73f2-47d5-9ae5-c82a0c9b4f4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:26:35.493391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 21 06:26:35.493391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: Jul 21 06:26:35.494349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0a210-73f2-47d5-9ae5-c82a0c9b4f4f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:26:35.505597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 750/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:26:35.513912 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe3df3-50d6-4389-bcfa-4d5bfd127d22 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:26:35.522998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe3df3-50d6-4389-bcfa-4d5bfd127d22 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:26:35.585211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:35.619863 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2808c5-5119-470f-a5d7-cca2a279de11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:26:35.620517 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2808c5-5119-470f-a5d7-cca2a279de11 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:35.620803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2808c5-5119-470f-a5d7-cca2a279de11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:35.621082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2808c5-5119-470f-a5d7-cca2a279de11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:35.621779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2808c5-5119-470f-a5d7-cca2a279de11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:26:35.623540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-5a2808c5-5119-470f-a5d7-cca2a279de11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:26:35.623540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 21 06:26:35.623540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: Jul 21 06:26:35.624726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2808c5-5119-470f-a5d7-cca2a279de11 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:26:35.678972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 747/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:26:35 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:26:35.683871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0a210-73f2-47d5-9ae5-c82a0c9b4f4f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:26:35.685453 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8ab0a210-73f2-47d5-9ae5-c82a0c9b4f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:35.718878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0a210-73f2-47d5-9ae5-c82a0c9b4f4f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:26:35.815248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39733659-9920-4639-bb03-e67aef8872d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:26:35.815835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39733659-9920-4639-bb03-e67aef8872d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:35.816119 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39733659-9920-4639-bb03-e67aef8872d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:35.816397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39733659-9920-4639-bb03-e67aef8872d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:35.817072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39733659-9920-4639-bb03-e67aef8872d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:26:35.818123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-39733659-9920-4639-bb03-e67aef8872d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:26:35.818123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 21 06:26:35.818123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: Jul 21 06:26:35.819320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39733659-9920-4639-bb03-e67aef8872d0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:26:35.976581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 751/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:26:35 2022] POST /identity/v3/users => generated 290 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:26:35.981773 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2808c5-5119-470f-a5d7-cca2a279de11 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:26:35.983323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5a2808c5-5119-470f-a5d7-cca2a279de11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:36.048115 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2808c5-5119-470f-a5d7-cca2a279de11 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:26:36.132103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:36.143992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277d07c-145d-4c5e-b898-1bd74b4e83ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:26:36.144664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277d07c-145d-4c5e-b898-1bd74b4e83ee admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:36.144945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277d07c-145d-4c5e-b898-1bd74b4e83ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:36.145218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277d07c-145d-4c5e-b898-1bd74b4e83ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:36.145957 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277d07c-145d-4c5e-b898-1bd74b4e83ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:26:36.147002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-2277d07c-145d-4c5e-b898-1bd74b4e83ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:26:36.147002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 21 06:26:36.147002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: Jul 21 06:26:36.147884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277d07c-145d-4c5e-b898-1bd74b4e83ee admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:26:36.161488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 748/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:26:36.165392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39733659-9920-4639-bb03-e67aef8872d0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:26:36.167510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-39733659-9920-4639-bb03-e67aef8872d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:36.178729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39733659-9920-4639-bb03-e67aef8872d0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:26:36.203536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:36.276914 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bba304-c3bb-4e66-a1cb-416a8df90ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:26:36.277579 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bba304-c3bb-4e66-a1cb-416a8df90ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:36.277863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bba304-c3bb-4e66-a1cb-416a8df90ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:36.278168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bba304-c3bb-4e66-a1cb-416a8df90ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:36.279156 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bba304-c3bb-4e66-a1cb-416a8df90ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:26:36.280161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-b0bba304-c3bb-4e66-a1cb-416a8df90ac3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:26:36.280161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 21 06:26:36.280161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: Jul 21 06:26:36.281037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bba304-c3bb-4e66-a1cb-416a8df90ac3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:26:36.294003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 752/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:26:36.298217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277d07c-145d-4c5e-b898-1bd74b4e83ee admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:26:36.299744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2277d07c-145d-4c5e-b898-1bd74b4e83ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:36.332727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277d07c-145d-4c5e-b898-1bd74b4e83ee admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:26:36.425699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142ca9c-3f7f-4cf5-aece-18bdfa80a0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:26:36.426557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142ca9c-3f7f-4cf5-aece-18bdfa80a0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:36.426842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142ca9c-3f7f-4cf5-aece-18bdfa80a0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:36.427122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142ca9c-3f7f-4cf5-aece-18bdfa80a0ba admin admin] PATH_INFO: `/v3/projects/071daf9b47214f63b3710a35b735be85/users/6ba4c1fcc0ac4a71abe705773df4ec4b/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:36.449277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142ca9c-3f7f-4cf5-aece-18bdfa80a0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=071daf9b47214f63b3710a35b735be85, user_id=6ba4c1fcc0ac4a71abe705773df4ec4b, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:26:36.450348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-c142ca9c-3f7f-4cf5-aece-18bdfa80a0ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:26:36.450348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 21 06:26:36.450348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: Jul 21 06:26:36.452273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142ca9c-3f7f-4cf5-aece-18bdfa80a0ba admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:26:36.477008 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 749/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:26:36 2022] PUT /identity/v3/projects/071daf9b47214f63b3710a35b735be85/users/6ba4c1fcc0ac4a71abe705773df4ec4b/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:26:36.489254 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c7da6-c455-4843-ba2c-2766664d5ee5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:36.489542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c7da6-c455-4843-ba2c-2766664d5ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:36.489819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c7da6-c455-4843-ba2c-2766664d5ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:36.546475 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a47c7da6-c455-4843-ba2c-2766664d5ee5 None None] MFA Rules not processed for user `6ba4c1fcc0ac4a71abe705773df4ec4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:26:36.547691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a47c7da6-c455-4843-ba2c-2766664d5ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:36.625141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 753/1502] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:26:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:26:36.639354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0596e-7a8d-4a23-a0ec-ec56ad21c6b8 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:36.639649 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0596e-7a8d-4a23-a0ec-ec56ad21c6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:36.639932 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0596e-7a8d-4a23-a0ec-ec56ad21c6b8 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:36.641313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 750/1503] 10.222.0.42 () {54 vars in 964 bytes} [Thu Jul 21 06:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:26:36.646419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e57fe8-2914-4943-87b8-b0a26c7b904c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:36.646710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e57fe8-2914-4943-87b8-b0a26c7b904c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:36.646991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e57fe8-2914-4943-87b8-b0a26c7b904c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:36.677377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:36.721894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-62e57fe8-2914-4943-87b8-b0a26c7b904c None None] MFA Rules not processed for user `710c422b8024403da666ae5054ba1289`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:26:36.723103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-62e57fe8-2914-4943-87b8-b0a26c7b904c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:36.800769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 754/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Jul 21 06:26:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:26:36.809117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0596e-7a8d-4a23-a0ec-ec56ad21c6b8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:26:36.810666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-54b0596e-7a8d-4a23-a0ec-ec56ad21c6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:36.861394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:36.916915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0596e-7a8d-4a23-a0ec-ec56ad21c6b8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:26:36.989370 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dfa91-e1bc-421e-ad30-e8143be70b3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:26:36.990050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dfa91-e1bc-421e-ad30-e8143be70b3d service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:36.990342 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dfa91-e1bc-421e-ad30-e8143be70b3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:36.990624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dfa91-e1bc-421e-ad30-e8143be70b3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:36.992260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a18dfa91-e1bc-421e-ad30-e8143be70b3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:37.090843 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dfa91-e1bc-421e-ad30-e8143be70b3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:26:37.092277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dfa91-e1bc-421e-ad30-e8143be70b3d service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:26:37.095839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:37.110943 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 751/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:26:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:26:37.152700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a85827-3264-4e22-a849-122a0628b1b9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:37.152985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a85827-3264-4e22-a849-122a0628b1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:37.153260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a85827-3264-4e22-a849-122a0628b1b9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:37.154819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 755/1506] 10.222.0.42 () {52 vars in 829 bytes} [Thu Jul 21 06:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:26:37.159356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dfa91-e1bc-421e-ad30-e8143be70b3d service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:26:37.160560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a18dfa91-e1bc-421e-ad30-e8143be70b3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:37.259103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dfa91-e1bc-421e-ad30-e8143be70b3d service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:26:37.351808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:26:37.352683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:37.352965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:37.353775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] PATH_INFO: `/v3/users/6ba4c1fcc0ac4a71abe705773df4ec4b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:37.366084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ba4c1fcc0ac4a71abe705773df4ec4b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:26:37.367118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:26:37.367118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 21 06:26:37.367118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: Jul 21 06:26:37.368320 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:26:37.378830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:37.431624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ba4c1fcc0ac4a71abe705773df4ec4b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:26:37.451696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ba4c1fcc0ac4a71abe705773df4ec4b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:26:37.464183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ba4c1fcc0ac4a71abe705773df4ec4b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:26:37.464739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ba4c1fcc0ac4a71abe705773df4ec4b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:26:37.465160 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ba4c1fcc0ac4a71abe705773df4ec4b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:26:37.479886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 752/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:26:37 2022] DELETE /identity/v3/users/6ba4c1fcc0ac4a71abe705773df4ec4b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:26:37.484429 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec92e73-aafa-45e3-903f-1f3c5e1b77d9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:37.484924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec92e73-aafa-45e3-903f-1f3c5e1b77d9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:37.485032 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec92e73-aafa-45e3-903f-1f3c5e1b77d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:37.539989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6ec92e73-aafa-45e3-903f-1f3c5e1b77d9 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:26:37.542219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6ec92e73-aafa-45e3-903f-1f3c5e1b77d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:37.616768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:37.630343 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 756/1508] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:26:37.641059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:26:37.642311 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:37.740873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169f9ef-f95b-42dd-989e-f7b259c74571 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:26:37.795907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:37.817027 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cae97b-023f-4e86-8dfd-ad884847ebdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:26:37.817681 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cae97b-023f-4e86-8dfd-ad884847ebdc service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:37.817989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cae97b-023f-4e86-8dfd-ad884847ebdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:37.818270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cae97b-023f-4e86-8dfd-ad884847ebdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:37.819899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-97cae97b-023f-4e86-8dfd-ad884847ebdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:37.927532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cae97b-023f-4e86-8dfd-ad884847ebdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:26:37.928959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cae97b-023f-4e86-8dfd-ad884847ebdc service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:26:37.947619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 753/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:26:38.294093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec92e73-aafa-45e3-903f-1f3c5e1b77d9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:26:38.295455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6ec92e73-aafa-45e3-903f-1f3c5e1b77d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:26:38.308482 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec92e73-aafa-45e3-903f-1f3c5e1b77d9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:26:38.408281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:26:38.409176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:26:38.409460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:26:38.409738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] PATH_INFO: `/v3/projects/071daf9b47214f63b3710a35b735be85` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:26:38.419650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] RBAC: Authorizing `identity:delete_project(project_id=071daf9b47214f63b3710a35b735be85)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:26:38.420692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:26:38.420692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 21 06:26:38.420692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: Jul 21 06:26:38.421880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:26:38.478424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '071daf9b47214f63b3710a35b735be85', 'request_id': 'req-49d958d3-478e-498f-9ff4-12a9841c881a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:26:38.484461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '071daf9b47214f63b3710a35b735be85', 'request_id': 'req-49d958d3-478e-498f-9ff4-12a9841c881a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:26:38.498065 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '071daf9b47214f63b3710a35b735be85', 'request_id': 'req-49d958d3-478e-498f-9ff4-12a9841c881a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:26:38.525271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] The token cache is being invalidate because project 071daf9b47214f63b3710a35b735be85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:26:38.526046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:26:38.526571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-49d958d3-478e-498f-9ff4-12a9841c881a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:26:38.527967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 757/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:26:38 2022] DELETE /identity/v3/projects/071daf9b47214f63b3710a35b735be85 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:26:47.944879 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=62 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:26:48.523333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:28:12.735491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378c3d0-57e3-429b-9c25-d69fca8db0f0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:12.736129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378c3d0-57e3-429b-9c25-d69fca8db0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:12.736129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378c3d0-57e3-429b-9c25-d69fca8db0f0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:12.737462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:28:12.742744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d962677-1ac4-44aa-b8b9-822acf1414e8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:12.743259 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d962677-1ac4-44aa-b8b9-822acf1414e8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:12.743362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d962677-1ac4-44aa-b8b9-822acf1414e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:12.824893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4d962677-1ac4-44aa-b8b9-822acf1414e8 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:28:12.826142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4d962677-1ac4-44aa-b8b9-822acf1414e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:12.930847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:28:13.094918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378c3d0-57e3-429b-9c25-d69fca8db0f0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:28:13.097005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a378c3d0-57e3-429b-9c25-d69fca8db0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:13.216568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378c3d0-57e3-429b-9c25-d69fca8db0f0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:28:13.223131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4945a1-e866-46e3-8c69-bc0971b7e8f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:28:13.223778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4945a1-e866-46e3-8c69-bc0971b7e8f9 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:13.224060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4945a1-e866-46e3-8c69-bc0971b7e8f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:13.224335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4945a1-e866-46e3-8c69-bc0971b7e8f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:13.232315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4945a1-e866-46e3-8c69-bc0971b7e8f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:28:13.234338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4945a1-e866-46e3-8c69-bc0971b7e8f9 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:28:13.243578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 755/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:28:13.872807 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffbde3-424d-4cc2-9a39-5275c029c94a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:13.873333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffbde3-424d-4cc2-9a39-5275c029c94a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:13.873333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffbde3-424d-4cc2-9a39-5275c029c94a None None] PATH_INFO: `/v3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:13.874947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 759/1514] 10.222.0.36 () {54 vars in 925 bytes} [Thu Jul 21 06:28:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:28:13.881708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aaeb9-dcbd-43e8-b4b5-5fab7766ae25 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:13.882143 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aaeb9-dcbd-43e8-b4b5-5fab7766ae25 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:13.882339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aaeb9-dcbd-43e8-b4b5-5fab7766ae25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:13.943064 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-412aaeb9-dcbd-43e8-b4b5-5fab7766ae25 None None] MFA Rules not processed for user `1da2d85eb18049f196aeefc51d3f1e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:28:13.944266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-412aaeb9-dcbd-43e8-b4b5-5fab7766ae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:14.012227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:28:14.018500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 756/1515] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Jul 21 06:28:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:28:14.072050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ba4a6-306e-42d2-be7b-3ccfc4112e22 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:14.072341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ba4a6-306e-42d2-be7b-3ccfc4112e22 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:14.072624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ba4a6-306e-42d2-be7b-3ccfc4112e22 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:14.074085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 760/1516] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:28:14.078817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cd769-8d34-4403-a36b-549632a95cb3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:14.079109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cd769-8d34-4403-a36b-549632a95cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:14.079389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cd769-8d34-4403-a36b-549632a95cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:14.097461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bf4cd769-8d34-4403-a36b-549632a95cb3 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:28:14.098689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bf4cd769-8d34-4403-a36b-549632a95cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:14.106631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:28:14.261051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ba4a6-306e-42d2-be7b-3ccfc4112e22 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:28:14.275488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ba4a6-306e-42d2-be7b-3ccfc4112e22 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:28:14.281529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f43dbb-5f4d-462c-9ef7-87c94ff50ec9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:28:14.282226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f43dbb-5f4d-462c-9ef7-87c94ff50ec9 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:14.282511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f43dbb-5f4d-462c-9ef7-87c94ff50ec9 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:14.282787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f43dbb-5f4d-462c-9ef7-87c94ff50ec9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:14.291081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f43dbb-5f4d-462c-9ef7-87c94ff50ec9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:28:14.292678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f43dbb-5f4d-462c-9ef7-87c94ff50ec9 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:28:14.302776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 761/1518] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:28:22.825901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307cb38-7833-416e-a129-489ff3a25aef None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:22.826433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307cb38-7833-416e-a129-489ff3a25aef None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:22.826433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307cb38-7833-416e-a129-489ff3a25aef None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:22.827809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 758/1519] 10.222.0.42 () {54 vars in 961 bytes} [Thu Jul 21 06:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:28:22.833636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ebdab-40f6-4c54-8b69-d61b0eb28c45 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:22.834075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ebdab-40f6-4c54-8b69-d61b0eb28c45 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:22.834242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ebdab-40f6-4c54-8b69-d61b0eb28c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:22.852582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e18ebdab-40f6-4c54-8b69-d61b0eb28c45 None None] MFA Rules not processed for user `1da2d85eb18049f196aeefc51d3f1e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:28:22.853765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e18ebdab-40f6-4c54-8b69-d61b0eb28c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:22.862433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 762/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Jul 21 06:28:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:28:22.869652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307cb38-7833-416e-a129-489ff3a25aef None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:28:22.883137 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307cb38-7833-416e-a129-489ff3a25aef None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:28:22.888093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd770391-905d-4df5-8208-73ad199f71fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da2d85eb18049f196aeefc51d3f1e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:28:22.888728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd770391-905d-4df5-8208-73ad199f71fa service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:22.889016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd770391-905d-4df5-8208-73ad199f71fa service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:22.889301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd770391-905d-4df5-8208-73ad199f71fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:22.898286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd770391-905d-4df5-8208-73ad199f71fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:28:22.899862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd770391-905d-4df5-8208-73ad199f71fa service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:28:22.910341 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 759/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:28:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:28:25.547289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c3768-0e2d-4c1e-8dad-203fd51b808e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:25.547828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c3768-0e2d-4c1e-8dad-203fd51b808e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:25.547828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c3768-0e2d-4c1e-8dad-203fd51b808e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:25.549206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:28:25.555070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1540a-4ae4-4ceb-832a-df95939be958 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:25.555497 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1540a-4ae4-4ceb-832a-df95939be958 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:25.555638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1540a-4ae4-4ceb-832a-df95939be958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:25.573368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a8a1540a-4ae4-4ceb-832a-df95939be958 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:28:25.574610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a8a1540a-4ae4-4ceb-832a-df95939be958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:25.585736 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:28:25.738831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c3768-0e2d-4c1e-8dad-203fd51b808e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:28:25.753431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c3768-0e2d-4c1e-8dad-203fd51b808e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:28:25.758794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6cf23-6984-4481-9ee6-97a7aa6723ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:28:25.759431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6cf23-6984-4481-9ee6-97a7aa6723ef service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:25.759710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6cf23-6984-4481-9ee6-97a7aa6723ef service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:25.759985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6cf23-6984-4481-9ee6-97a7aa6723ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:25.768080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6cf23-6984-4481-9ee6-97a7aa6723ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:28:25.769656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6cf23-6984-4481-9ee6-97a7aa6723ef service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:28:25.778989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 764/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:28:26.598777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20c219-a8b7-4dd8-93c4-0480b8b276d1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:26.599345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20c219-a8b7-4dd8-93c4-0480b8b276d1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:26.599345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20c219-a8b7-4dd8-93c4-0480b8b276d1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:26.600728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 06:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:28:26.605474 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502cc28-a729-44d4-a7b5-cc3b20f6ec93 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:26.605885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502cc28-a729-44d4-a7b5-cc3b20f6ec93 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:26.606057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502cc28-a729-44d4-a7b5-cc3b20f6ec93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:26.624298 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f502cc28-a729-44d4-a7b5-cc3b20f6ec93 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:28:26.626952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f502cc28-a729-44d4-a7b5-cc3b20f6ec93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:26.634836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 06:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:28:26.786192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20c219-a8b7-4dd8-93c4-0480b8b276d1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:28:26.800488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20c219-a8b7-4dd8-93c4-0480b8b276d1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:28:26.805917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4ed93-96e1-48b2-ab6a-513a9cd24156 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:28:26.806579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4ed93-96e1-48b2-ab6a-513a9cd24156 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:26.806865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4ed93-96e1-48b2-ab6a-513a9cd24156 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:26.807149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4ed93-96e1-48b2-ab6a-513a9cd24156 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:26.815182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4ed93-96e1-48b2-ab6a-513a9cd24156 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:28:26.816759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4ed93-96e1-48b2-ab6a-513a9cd24156 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:28:26.826238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 762/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:28:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:28:30.761797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80f6f2-be3f-40f1-ad07-199ba84c5e49 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:30.762451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80f6f2-be3f-40f1-ad07-199ba84c5e49 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:30.762451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80f6f2-be3f-40f1-ad07-199ba84c5e49 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:30.763752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:28:30.769913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bfb14-c18b-412c-b75a-972e89940013 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:30.770397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bfb14-c18b-412c-b75a-972e89940013 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:30.770505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bfb14-c18b-412c-b75a-972e89940013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:30.788557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-125bfb14-c18b-412c-b75a-972e89940013 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:28:30.789740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-125bfb14-c18b-412c-b75a-972e89940013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:30.797658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:28:30.871792 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80f6f2-be3f-40f1-ad07-199ba84c5e49 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:28:30.873514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0b80f6f2-be3f-40f1-ad07-199ba84c5e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:30.905828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:28:30.952055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80f6f2-be3f-40f1-ad07-199ba84c5e49 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:28:30.955852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d59c3-b67b-4f7a-9f7d-8c6d090d4ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:28:30.956496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d59c3-b67b-4f7a-9f7d-8c6d090d4ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:30.956780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d59c3-b67b-4f7a-9f7d-8c6d090d4ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:30.957057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d59c3-b67b-4f7a-9f7d-8c6d090d4ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:30.965975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d59c3-b67b-4f7a-9f7d-8c6d090d4ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:28:30.967408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d59c3-b67b-4f7a-9f7d-8c6d090d4ab2 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:28:30.977296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 767/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:28:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:28:33.017396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b7065-9881-4a1a-9dfe-3ed16a1d0f2d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:33.017989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b7065-9881-4a1a-9dfe-3ed16a1d0f2d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:33.017989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b7065-9881-4a1a-9dfe-3ed16a1d0f2d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:33.019376 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:28:33.024454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512347c-e6bc-4510-b107-0f9fea5fee3d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:33.024844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512347c-e6bc-4510-b107-0f9fea5fee3d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:33.025028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512347c-e6bc-4510-b107-0f9fea5fee3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:33.043112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4512347c-e6bc-4510-b107-0f9fea5fee3d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:28:33.044315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4512347c-e6bc-4510-b107-0f9fea5fee3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:33.052676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:28:33.094593 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b7065-9881-4a1a-9dfe-3ed16a1d0f2d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:28:33.105241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b7065-9881-4a1a-9dfe-3ed16a1d0f2d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:28:33.109018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b69da4-375e-4d07-af54-2c1f32b6e8dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:28:33.109703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b69da4-375e-4d07-af54-2c1f32b6e8dd service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:33.110027 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b69da4-375e-4d07-af54-2c1f32b6e8dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:33.110315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b69da4-375e-4d07-af54-2c1f32b6e8dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:33.119262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b69da4-375e-4d07-af54-2c1f32b6e8dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:28:33.120680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b69da4-375e-4d07-af54-2c1f32b6e8dd service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:28:33.130981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 765/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:28:36.679740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fabd4-3fd3-4248-a91f-e41ddb1ad9d7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:36.680333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fabd4-3fd3-4248-a91f-e41ddb1ad9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:36.680333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fabd4-3fd3-4248-a91f-e41ddb1ad9d7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:36.681666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:28:36.687099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2138d-4730-45f6-a1e4-37505f5c04fe None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:36.687551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2138d-4730-45f6-a1e4-37505f5c04fe None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:36.687701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2138d-4730-45f6-a1e4-37505f5c04fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:36.706683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-15a2138d-4730-45f6-a1e4-37505f5c04fe None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:28:36.707872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-15a2138d-4730-45f6-a1e4-37505f5c04fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:36.715732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:28:37.027192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fabd4-3fd3-4248-a91f-e41ddb1ad9d7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:28:37.028979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f84fabd4-3fd3-4248-a91f-e41ddb1ad9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:37.108104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fabd4-3fd3-4248-a91f-e41ddb1ad9d7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:28:37.112264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac8267-e32e-46a8-96e6-7fe928a08591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:28:37.112915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8267-e32e-46a8-96e6-7fe928a08591 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:37.113202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8267-e32e-46a8-96e6-7fe928a08591 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:37.113483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8267-e32e-46a8-96e6-7fe928a08591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:37.122471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac8267-e32e-46a8-96e6-7fe928a08591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:28:37.123910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac8267-e32e-46a8-96e6-7fe928a08591 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:28:37.134479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 770/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:28:39.212203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c16716-3bdb-49ac-ae8b-e24b5cebaa7d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:39.212768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c16716-3bdb-49ac-ae8b-e24b5cebaa7d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:39.212768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c16716-3bdb-49ac-ae8b-e24b5cebaa7d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:39.214279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 06:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:28:39.219291 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857195d0-1048-4975-93d6-fb7fa4182b6f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:39.219732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857195d0-1048-4975-93d6-fb7fa4182b6f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:39.219876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857195d0-1048-4975-93d6-fb7fa4182b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:39.238623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-857195d0-1048-4975-93d6-fb7fa4182b6f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:28:39.239807 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-857195d0-1048-4975-93d6-fb7fa4182b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:39.247517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 06:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:28:39.546656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c16716-3bdb-49ac-ae8b-e24b5cebaa7d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:28:39.548436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d6c16716-3bdb-49ac-ae8b-e24b5cebaa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:28:39.561457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c16716-3bdb-49ac-ae8b-e24b5cebaa7d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:28:39.565185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59c524-6038-4c9a-a857-65678001ce6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:28:39.565824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59c524-6038-4c9a-a857-65678001ce6b service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:28:39.566141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59c524-6038-4c9a-a857-65678001ce6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:28:39.566514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59c524-6038-4c9a-a857-65678001ce6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:28:39.575296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59c524-6038-4c9a-a857-65678001ce6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:28:39.576714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59c524-6038-4c9a-a857-65678001ce6b service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:28:39.586713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 768/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:28:49.233816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=63 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:28:49.582597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=37 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:07.701000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c50b36-0dbb-4cf6-827b-387ab1c1bccf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:07.702188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c50b36-0dbb-4cf6-827b-387ab1c1bccf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:07.702698 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c50b36-0dbb-4cf6-827b-387ab1c1bccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:07.728533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f4c50b36-0dbb-4cf6-827b-387ab1c1bccf None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:07.729772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f4c50b36-0dbb-4cf6-827b-387ab1c1bccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:07.747949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 772/1540] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:07.757015 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aac80d-4156-41bd-bb66-f589180dfd38 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:07.758829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aac80d-4156-41bd-bb66-f589180dfd38 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:07.759379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aac80d-4156-41bd-bb66-f589180dfd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:07.799550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-67aac80d-4156-41bd-bb66-f589180dfd38 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:07.800797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-67aac80d-4156-41bd-bb66-f589180dfd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:07.816599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 769/1541] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:07.827134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c50b36-0dbb-4cf6-827b-387ab1c1bccf None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:07.837838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c50b36-0dbb-4cf6-827b-387ab1c1bccf None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:07.859994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7447e-e7d1-4ae2-a057-3e7e9a8c10ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:07.860696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7447e-e7d1-4ae2-a057-3e7e9a8c10ce admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:07.860984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7447e-e7d1-4ae2-a057-3e7e9a8c10ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:07.861266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7447e-e7d1-4ae2-a057-3e7e9a8c10ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:07.862051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7447e-e7d1-4ae2-a057-3e7e9a8c10ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:07.862371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7447e-e7d1-4ae2-a057-3e7e9a8c10ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:07.863995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7447e-e7d1-4ae2-a057-3e7e9a8c10ce admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:07.885372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 773/1542] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:07.893986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aac80d-4156-41bd-bb66-f589180dfd38 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:07.910740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aac80d-4156-41bd-bb66-f589180dfd38 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:07.925728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d0c76-f430-449f-a0c3-9e08f6192d34 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:07.926045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d0c76-f430-449f-a0c3-9e08f6192d34 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:07.926289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d0c76-f430-449f-a0c3-9e08f6192d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:07.927103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149a9d2-6cc3-4372-9f7a-6ffdb900491a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:07.928045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149a9d2-6cc3-4372-9f7a-6ffdb900491a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:07.928547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149a9d2-6cc3-4372-9f7a-6ffdb900491a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:07.929040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149a9d2-6cc3-4372-9f7a-6ffdb900491a admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:07.929977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149a9d2-6cc3-4372-9f7a-6ffdb900491a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:07.932494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149a9d2-6cc3-4372-9f7a-6ffdb900491a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:07.966848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-815d0c76-f430-449f-a0c3-9e08f6192d34 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:07.968076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-815d0c76-f430-449f-a0c3-9e08f6192d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:07.988886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 774/1543] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:30:08.000089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802719bf-becc-4385-bf50-48a8fcc4b771 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.000089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802719bf-becc-4385-bf50-48a8fcc4b771 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.000089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802719bf-becc-4385-bf50-48a8fcc4b771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.006589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 770/1544] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:07 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:08.010372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149a9d2-6cc3-4372-9f7a-6ffdb900491a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.031470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-802719bf-becc-4385-bf50-48a8fcc4b771 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.032774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-802719bf-becc-4385-bf50-48a8fcc4b771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.124533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 775/1545] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:30:08.125873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987f717-7fb1-436d-9483-178e7116749d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.126192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987f717-7fb1-436d-9483-178e7116749d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.126477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987f717-7fb1-436d-9483-178e7116749d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.138583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149a9d2-6cc3-4372-9f7a-6ffdb900491a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.156708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd95a3-6b2b-4e46-b231-fc5262d6dd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.157283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd95a3-6b2b-4e46-b231-fc5262d6dd5c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.157569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd95a3-6b2b-4e46-b231-fc5262d6dd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.157846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd95a3-6b2b-4e46-b231-fc5262d6dd5c admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.158572 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd95a3-6b2b-4e46-b231-fc5262d6dd5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.159591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-6efd95a3-6b2b-4e46-b231-fc5262d6dd5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:30:08.159591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 21 06:30:08.159591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: Jul 21 06:30:08.161049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd95a3-6b2b-4e46-b231-fc5262d6dd5c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.161251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5987f717-7fb1-436d-9483-178e7116749d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.166572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5987f717-7fb1-436d-9483-178e7116749d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.196548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 776/1546] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.197496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987f717-7fb1-436d-9483-178e7116749d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.211843 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 771/1547] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:08.214824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38709b9-f69e-432d-ae29-1095491c909a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.215696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38709b9-f69e-432d-ae29-1095491c909a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.215992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38709b9-f69e-432d-ae29-1095491c909a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.225660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987f717-7fb1-436d-9483-178e7116749d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.236507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605517c3-a355-4e26-86e9-4f2e10bd0966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.237017 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605517c3-a355-4e26-86e9-4f2e10bd0966 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.239887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605517c3-a355-4e26-86e9-4f2e10bd0966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.240185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605517c3-a355-4e26-86e9-4f2e10bd0966 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.240928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605517c3-a355-4e26-86e9-4f2e10bd0966 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:08.242282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605517c3-a355-4e26-86e9-4f2e10bd0966 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.246777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605517c3-a355-4e26-86e9-4f2e10bd0966 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.254499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e38709b9-f69e-432d-ae29-1095491c909a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.256189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e38709b9-f69e-432d-ae29-1095491c909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.263482 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 777/1548] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:08.264946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c5d26-2be1-4b72-a819-180ceb816315 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.265237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c5d26-2be1-4b72-a819-180ceb816315 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.265516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c5d26-2be1-4b72-a819-180ceb816315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.266925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 772/1549] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.268255 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238599e0-f356-4a65-8a95-c284002e5dce None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.268614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238599e0-f356-4a65-8a95-c284002e5dce None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.268962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238599e0-f356-4a65-8a95-c284002e5dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.289932 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c60c5d26-2be1-4b72-a819-180ceb816315 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.290285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-238599e0-f356-4a65-8a95-c284002e5dce None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.291479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-238599e0-f356-4a65-8a95-c284002e5dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.292127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c60c5d26-2be1-4b72-a819-180ceb816315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.303753 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 773/1550] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.304579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238599e0-f356-4a65-8a95-c284002e5dce None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.306503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 778/1551] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.312836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20969bfe-07b8-4df9-b861-3ac5d8904433 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.312836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20969bfe-07b8-4df9-b861-3ac5d8904433 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.312836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20969bfe-07b8-4df9-b861-3ac5d8904433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.315155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238599e0-f356-4a65-8a95-c284002e5dce None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.320160 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d3c8e-2eab-43a4-96a3-1e777e3da354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.320832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d3c8e-2eab-43a4-96a3-1e777e3da354 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.321153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d3c8e-2eab-43a4-96a3-1e777e3da354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.321435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d3c8e-2eab-43a4-96a3-1e777e3da354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.322227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d3c8e-2eab-43a4-96a3-1e777e3da354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.323964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d3c8e-2eab-43a4-96a3-1e777e3da354 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.329612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-20969bfe-07b8-4df9-b861-3ac5d8904433 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.330875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-20969bfe-07b8-4df9-b861-3ac5d8904433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.334281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 774/1552] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:08.335493 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd9a53-9f84-481d-8e3a-59c44141acf3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.335787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd9a53-9f84-481d-8e3a-59c44141acf3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.336069 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd9a53-9f84-481d-8e3a-59c44141acf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.339309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 779/1553] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.340728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20969bfe-07b8-4df9-b861-3ac5d8904433 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.349977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20969bfe-07b8-4df9-b861-3ac5d8904433 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.354582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33651d9-220a-4f22-9d8a-66a65d499908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.355228 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33651d9-220a-4f22-9d8a-66a65d499908 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.355348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c5bd9a53-9f84-481d-8e3a-59c44141acf3 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.356235 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33651d9-220a-4f22-9d8a-66a65d499908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.356813 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33651d9-220a-4f22-9d8a-66a65d499908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.357116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33651d9-220a-4f22-9d8a-66a65d499908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.357332 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c5bd9a53-9f84-481d-8e3a-59c44141acf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.358165 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-e33651d9-220a-4f22-9d8a-66a65d499908 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:30:08.358165 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 21 06:30:08.358165 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: Jul 21 06:30:08.359334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33651d9-220a-4f22-9d8a-66a65d499908 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.366515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 775/1554] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.367566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08847e2-c27a-4194-8afa-847ba198dd32 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.367862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08847e2-c27a-4194-8afa-847ba198dd32 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.368142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08847e2-c27a-4194-8afa-847ba198dd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.386340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b08847e2-c27a-4194-8afa-847ba198dd32 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.387531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 780/1555] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:08.387663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b08847e2-c27a-4194-8afa-847ba198dd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.388370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33651d9-220a-4f22-9d8a-66a65d499908 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.448972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 776/1556] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.451022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb6834-c6ce-4443-b8df-68ded274024b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.451314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb6834-c6ce-4443-b8df-68ded274024b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.451592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb6834-c6ce-4443-b8df-68ded274024b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.451716 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33651d9-220a-4f22-9d8a-66a65d499908 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.456585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff59f4-ac80-4714-8025-9002ff94f866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.457252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff59f4-ac80-4714-8025-9002ff94f866 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.457724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff59f4-ac80-4714-8025-9002ff94f866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.458039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff59f4-ac80-4714-8025-9002ff94f866 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.458778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff59f4-ac80-4714-8025-9002ff94f866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:08.459087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff59f4-ac80-4714-8025-9002ff94f866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.460702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff59f4-ac80-4714-8025-9002ff94f866 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.471035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-88cb6834-c6ce-4443-b8df-68ded274024b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.472241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-88cb6834-c6ce-4443-b8df-68ded274024b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.473501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 781/1557] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:08.475237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71d265-5c69-4828-8783-4a8b8d323a70 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.475733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71d265-5c69-4828-8783-4a8b8d323a70 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.476079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71d265-5c69-4828-8783-4a8b8d323a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.486086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 777/1558] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.486086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb6834-c6ce-4443-b8df-68ded274024b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.495346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3b71d265-5c69-4828-8783-4a8b8d323a70 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.496180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb6834-c6ce-4443-b8df-68ded274024b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.497010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3b71d265-5c69-4828-8783-4a8b8d323a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.500986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27608010-a438-46c8-ad9a-9cbf3db1938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.501654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27608010-a438-46c8-ad9a-9cbf3db1938c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.501955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27608010-a438-46c8-ad9a-9cbf3db1938c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.502253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27608010-a438-46c8-ad9a-9cbf3db1938c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.502995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27608010-a438-46c8-ad9a-9cbf3db1938c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.504590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27608010-a438-46c8-ad9a-9cbf3db1938c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.506020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 782/1559] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.507144 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a852cd-c500-4b9c-86b3-4d1fbb0bc4aa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.507370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a852cd-c500-4b9c-86b3-4d1fbb0bc4aa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.507651 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a852cd-c500-4b9c-86b3-4d1fbb0bc4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.515844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 778/1560] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:08.517224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748636c-ad3e-485e-a7f8-cfb607b5c86e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.517520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748636c-ad3e-485e-a7f8-cfb607b5c86e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.517804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748636c-ad3e-485e-a7f8-cfb607b5c86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.521365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:08.526336 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:08.529488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-59a852cd-c500-4b9c-86b3-4d1fbb0bc4aa None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.531326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-59a852cd-c500-4b9c-86b3-4d1fbb0bc4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.538270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8748636c-ad3e-485e-a7f8-cfb607b5c86e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.539442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8748636c-ad3e-485e-a7f8-cfb607b5c86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.539613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 783/1561] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.540370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a852cd-c500-4b9c-86b3-4d1fbb0bc4aa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.547878 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 779/1562] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.548685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748636c-ad3e-485e-a7f8-cfb607b5c86e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.550581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a852cd-c500-4b9c-86b3-4d1fbb0bc4aa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.555143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0df0c-0c99-45e1-bd2f-46cc7365c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.555812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0df0c-0c99-45e1-bd2f-46cc7365c20f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.556093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0df0c-0c99-45e1-bd2f-46cc7365c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.556366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0df0c-0c99-45e1-bd2f-46cc7365c20f admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.557048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0df0c-0c99-45e1-bd2f-46cc7365c20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.558040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748636c-ad3e-485e-a7f8-cfb607b5c86e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.559129 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0df0c-0c99-45e1-bd2f-46cc7365c20f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.562684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002bfc4-89a6-4916-8629-0fd418a94265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.563339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002bfc4-89a6-4916-8629-0fd418a94265 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.563628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002bfc4-89a6-4916-8629-0fd418a94265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.563911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002bfc4-89a6-4916-8629-0fd418a94265 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.564646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002bfc4-89a6-4916-8629-0fd418a94265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:08.564961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002bfc4-89a6-4916-8629-0fd418a94265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.566001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-2002bfc4-89a6-4916-8629-0fd418a94265 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:30:08.566001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 21 06:30:08.566001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: Jul 21 06:30:08.567066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002bfc4-89a6-4916-8629-0fd418a94265 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.588163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 780/1563] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:08.588395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 784/1564] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/users => generated 284 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:08.588509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002bfc4-89a6-4916-8629-0fd418a94265 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.588612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948ec89-7c47-4a43-bc85-cd4408f3ee7e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.588612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948ec89-7c47-4a43-bc85-cd4408f3ee7e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.588612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948ec89-7c47-4a43-bc85-cd4408f3ee7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.600077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002bfc4-89a6-4916-8629-0fd418a94265 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.603246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c948ec89-7c47-4a43-bc85-cd4408f3ee7e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:08.604435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c948ec89-7c47-4a43-bc85-cd4408f3ee7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:08.604758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bcdff-6d79-406a-8ed5-cd16eca04d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.605438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bcdff-6d79-406a-8ed5-cd16eca04d43 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.605820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bcdff-6d79-406a-8ed5-cd16eca04d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.606142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bcdff-6d79-406a-8ed5-cd16eca04d43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.606840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bcdff-6d79-406a-8ed5-cd16eca04d43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.608777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bcdff-6d79-406a-8ed5-cd16eca04d43 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.616448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 785/1565] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:08.616448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948ec89-7c47-4a43-bc85-cd4408f3ee7e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.625403 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948ec89-7c47-4a43-bc85-cd4408f3ee7e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.630544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e453e5-d78f-4c5f-bda0-cd48c7b418f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.631151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e453e5-d78f-4c5f-bda0-cd48c7b418f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.631566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e453e5-d78f-4c5f-bda0-cd48c7b418f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.631884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e453e5-d78f-4c5f-bda0-cd48c7b418f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.632658 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e453e5-d78f-4c5f-bda0-cd48c7b418f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:08.633093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e453e5-d78f-4c5f-bda0-cd48c7b418f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.634811 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e453e5-d78f-4c5f-bda0-cd48c7b418f2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.641668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 781/1566] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/projects => generated 357 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:08.642576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bcdff-6d79-406a-8ed5-cd16eca04d43 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.657810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 786/1567] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:08.657810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e453e5-d78f-4c5f-bda0-cd48c7b418f2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.667410 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e453e5-d78f-4c5f-bda0-cd48c7b418f2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.704601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bcdff-6d79-406a-8ed5-cd16eca04d43 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.709320 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7585c12-41a7-4dad-a389-94d0b2abc225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.710241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7585c12-41a7-4dad-a389-94d0b2abc225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.710519 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7585c12-41a7-4dad-a389-94d0b2abc225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.710798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7585c12-41a7-4dad-a389-94d0b2abc225 admin admin] PATH_INFO: `/v3/projects/9cd8a1a642a24b519aefdde9ed336536/users/774dd66c674a4955a439afd30354efab/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.727691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca96bc9-5023-4e14-a7c4-1daca24efefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.729398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca96bc9-5023-4e14-a7c4-1daca24efefa admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.729781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca96bc9-5023-4e14-a7c4-1daca24efefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.730126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca96bc9-5023-4e14-a7c4-1daca24efefa admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.731416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca96bc9-5023-4e14-a7c4-1daca24efefa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:08.731716 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca96bc9-5023-4e14-a7c4-1daca24efefa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.732593 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7585c12-41a7-4dad-a389-94d0b2abc225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd8a1a642a24b519aefdde9ed336536, user_id=774dd66c674a4955a439afd30354efab, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.733373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca96bc9-5023-4e14-a7c4-1daca24efefa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.735362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7585c12-41a7-4dad-a389-94d0b2abc225 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.749141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 787/1568] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:08.750535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca96bc9-5023-4e14-a7c4-1daca24efefa admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.758131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 782/1569] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:08 2022] PUT /identity/v3/projects/9cd8a1a642a24b519aefdde9ed336536/users/774dd66c674a4955a439afd30354efab/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:08.760732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7585c12-41a7-4dad-a389-94d0b2abc225 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.767984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca96bc9-5023-4e14-a7c4-1daca24efefa admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.830592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911fc8a-104e-438c-a9c6-2f916af8aca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.831276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911fc8a-104e-438c-a9c6-2f916af8aca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.831558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911fc8a-104e-438c-a9c6-2f916af8aca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.831834 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911fc8a-104e-438c-a9c6-2f916af8aca6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.832565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911fc8a-104e-438c-a9c6-2f916af8aca6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:08.832872 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911fc8a-104e-438c-a9c6-2f916af8aca6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.834531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911fc8a-104e-438c-a9c6-2f916af8aca6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.835589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7585c12-41a7-4dad-a389-94d0b2abc225 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.841077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2705a-a256-4ed6-8033-0b5c5269cf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.841785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2705a-a256-4ed6-8033-0b5c5269cf9c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.842159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2705a-a256-4ed6-8033-0b5c5269cf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.842444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2705a-a256-4ed6-8033-0b5c5269cf9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.843190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2705a-a256-4ed6-8033-0b5c5269cf9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.844788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2705a-a256-4ed6-8033-0b5c5269cf9c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.849517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 788/1570] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:08.850594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911fc8a-104e-438c-a9c6-2f916af8aca6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.855857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 783/1571] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:08.856924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2705a-a256-4ed6-8033-0b5c5269cf9c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.869006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911fc8a-104e-438c-a9c6-2f916af8aca6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.869343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2705a-a256-4ed6-8033-0b5c5269cf9c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.874854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4f0dc-06cf-42ed-982e-cb78f6997057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.875308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbfade-8bdf-46c9-beb2-7b79a7ac5ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.875518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4f0dc-06cf-42ed-982e-cb78f6997057 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.875799 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4f0dc-06cf-42ed-982e-cb78f6997057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.876074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4f0dc-06cf-42ed-982e-cb78f6997057 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.876170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbfade-8bdf-46c9-beb2-7b79a7ac5ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.876404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbfade-8bdf-46c9-beb2-7b79a7ac5ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.876686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbfade-8bdf-46c9-beb2-7b79a7ac5ff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.876802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4f0dc-06cf-42ed-982e-cb78f6997057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.877470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbfade-8bdf-46c9-beb2-7b79a7ac5ff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:08.877770 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbfade-8bdf-46c9-beb2-7b79a7ac5ff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.878699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4f0dc-06cf-42ed-982e-cb78f6997057 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.879408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbfade-8bdf-46c9-beb2-7b79a7ac5ff8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.893991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 784/1572] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:08.894622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbfade-8bdf-46c9-beb2-7b79a7ac5ff8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.907083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbfade-8bdf-46c9-beb2-7b79a7ac5ff8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.913403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f10eab-2496-44f8-9b53-0b77892a226d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:08.914172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f10eab-2496-44f8-9b53-0b77892a226d admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:08.914411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f10eab-2496-44f8-9b53-0b77892a226d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:08.914711 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f10eab-2496-44f8-9b53-0b77892a226d admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:08.915408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f10eab-2496-44f8-9b53-0b77892a226d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:08.916086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 789/1573] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:08.916911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4f0dc-06cf-42ed-982e-cb78f6997057 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.917608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f10eab-2496-44f8-9b53-0b77892a226d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:08.940475 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 785/1574] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/users => generated 279 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:08.941381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f10eab-2496-44f8-9b53-0b77892a226d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:08.952457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f10eab-2496-44f8-9b53-0b77892a226d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.956443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:08.961322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:08.994346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4f0dc-06cf-42ed-982e-cb78f6997057 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:08.999874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70f23f-5f38-4861-a2f8-c40f624ee081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.000541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70f23f-5f38-4861-a2f8-c40f624ee081 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.000823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70f23f-5f38-4861-a2f8-c40f624ee081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.001100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70f23f-5f38-4861-a2f8-c40f624ee081 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.001931 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70f23f-5f38-4861-a2f8-c40f624ee081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.004011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70f23f-5f38-4861-a2f8-c40f624ee081 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.016745 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7139574-fc02-4fd4-b182-c69f2645b2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.017421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7139574-fc02-4fd4-b182-c69f2645b2fa admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.017702 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7139574-fc02-4fd4-b182-c69f2645b2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.018010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7139574-fc02-4fd4-b182-c69f2645b2fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.018841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7139574-fc02-4fd4-b182-c69f2645b2fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.020750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7139574-fc02-4fd4-b182-c69f2645b2fa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.035388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 790/1575] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:09.038168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426606c-8bdb-4c52-9cdb-8384321c294a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.038986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426606c-8bdb-4c52-9cdb-8384321c294a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.039471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426606c-8bdb-4c52-9cdb-8384321c294a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.054495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 786/1576] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:08 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:09.055879 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7139574-fc02-4fd4-b182-c69f2645b2fa admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.062286 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8426606c-8bdb-4c52-9cdb-8384321c294a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:09.063499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8426606c-8bdb-4c52-9cdb-8384321c294a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:09.128730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7139574-fc02-4fd4-b182-c69f2645b2fa admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.132571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 791/1577] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:09.133425 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426606c-8bdb-4c52-9cdb-8384321c294a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.133538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e021fc9-f92f-41e3-9f08-c8d79b040d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.134251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e021fc9-f92f-41e3-9f08-c8d79b040d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.134538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e021fc9-f92f-41e3-9f08-c8d79b040d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.134819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e021fc9-f92f-41e3-9f08-c8d79b040d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.135515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e021fc9-f92f-41e3-9f08-c8d79b040d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.138933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e021fc9-f92f-41e3-9f08-c8d79b040d9c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.144600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426606c-8bdb-4c52-9cdb-8384321c294a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.150375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a7d25-3cbc-4676-b36b-317276d3f777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.151064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a7d25-3cbc-4676-b36b-317276d3f777 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.151347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a7d25-3cbc-4676-b36b-317276d3f777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.151622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a7d25-3cbc-4676-b36b-317276d3f777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.152382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a7d25-3cbc-4676-b36b-317276d3f777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.154051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a7d25-3cbc-4676-b36b-317276d3f777 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.165598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 792/1578] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.166443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a7d25-3cbc-4676-b36b-317276d3f777 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.172813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 787/1579] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:09 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:09.172813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e021fc9-f92f-41e3-9f08-c8d79b040d9c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.178193 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a7d25-3cbc-4676-b36b-317276d3f777 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.241010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3bc73-4651-48b1-a313-26f106fd5f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.241682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3bc73-4651-48b1-a313-26f106fd5f66 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.242009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3bc73-4651-48b1-a313-26f106fd5f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.242292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3bc73-4651-48b1-a313-26f106fd5f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.242985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3bc73-4651-48b1-a313-26f106fd5f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.244895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3bc73-4651-48b1-a313-26f106fd5f66 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.250742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e021fc9-f92f-41e3-9f08-c8d79b040d9c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.258068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408b901-1365-477c-9f45-d94594006b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.258843 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408b901-1365-477c-9f45-d94594006b24 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.259046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408b901-1365-477c-9f45-d94594006b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.259335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408b901-1365-477c-9f45-d94594006b24 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.260040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408b901-1365-477c-9f45-d94594006b24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.262573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408b901-1365-477c-9f45-d94594006b24 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.273670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 793/1580] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:09 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:09.274558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3bc73-4651-48b1-a313-26f106fd5f66 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.291698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 788/1581] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:09 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:09.292426 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408b901-1365-477c-9f45-d94594006b24 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.303510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408b901-1365-477c-9f45-d94594006b24 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.313559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:09.314397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:09.355278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3bc73-4651-48b1-a313-26f106fd5f66 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.360509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a251d-846f-4dcd-a495-78a00dd159fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.361184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a251d-846f-4dcd-a495-78a00dd159fa admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.361471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a251d-846f-4dcd-a495-78a00dd159fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.361747 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a251d-846f-4dcd-a495-78a00dd159fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.362520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a251d-846f-4dcd-a495-78a00dd159fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.364126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a251d-846f-4dcd-a495-78a00dd159fa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.377488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 794/1582] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.378359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a251d-846f-4dcd-a495-78a00dd159fa admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.382061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087817cb-222c-4293-a4d3-9e0e26b1f8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.382761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087817cb-222c-4293-a4d3-9e0e26b1f8ff admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.383045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087817cb-222c-4293-a4d3-9e0e26b1f8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.383323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087817cb-222c-4293-a4d3-9e0e26b1f8ff admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.384022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087817cb-222c-4293-a4d3-9e0e26b1f8ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.386266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087817cb-222c-4293-a4d3-9e0e26b1f8ff admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.390515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a251d-846f-4dcd-a495-78a00dd159fa admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.395417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63bb40-f85b-48ac-a9ee-e11b08747097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.396089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63bb40-f85b-48ac-a9ee-e11b08747097 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.396500 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63bb40-f85b-48ac-a9ee-e11b08747097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.396788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63bb40-f85b-48ac-a9ee-e11b08747097 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.397483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63bb40-f85b-48ac-a9ee-e11b08747097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.399805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63bb40-f85b-48ac-a9ee-e11b08747097 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.409281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 789/1583] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:09 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:09.410308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087817cb-222c-4293-a4d3-9e0e26b1f8ff admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.424766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 795/1584] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:09 2022] POST /identity/v3/users => generated 295 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:09.425418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63bb40-f85b-48ac-a9ee-e11b08747097 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.434801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63bb40-f85b-48ac-a9ee-e11b08747097 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.439892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134979f3-1470-4cd5-9a38-9b7beeb349b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.440894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134979f3-1470-4cd5-9a38-9b7beeb349b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.441186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134979f3-1470-4cd5-9a38-9b7beeb349b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.441471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134979f3-1470-4cd5-9a38-9b7beeb349b6 admin admin] PATH_INFO: `/v3/projects/c6346dc927ab4830b13d954cda6e048d/users/baeb650373e84766a47f0825032a6648/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.458871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134979f3-1470-4cd5-9a38-9b7beeb349b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6346dc927ab4830b13d954cda6e048d, user_id=baeb650373e84766a47f0825032a6648, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.459994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-134979f3-1470-4cd5-9a38-9b7beeb349b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:30:09.459994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 21 06:30:09.459994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: Jul 21 06:30:09.462151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134979f3-1470-4cd5-9a38-9b7beeb349b6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.471218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087817cb-222c-4293-a4d3-9e0e26b1f8ff admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.476493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 796/1585] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:09 2022] PUT /identity/v3/projects/c6346dc927ab4830b13d954cda6e048d/users/baeb650373e84766a47f0825032a6648/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:09.477360 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134979f3-1470-4cd5-9a38-9b7beeb349b6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.522254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7e8c0-367b-4514-afbb-34f9933fd4b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.523000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7e8c0-367b-4514-afbb-34f9933fd4b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.523262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7e8c0-367b-4514-afbb-34f9933fd4b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.523584 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7e8c0-367b-4514-afbb-34f9933fd4b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.554735 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134979f3-1470-4cd5-9a38-9b7beeb349b6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.560525 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a79900-0e5f-4bc3-a34c-0495dabdc1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.561200 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a79900-0e5f-4bc3-a34c-0495dabdc1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.561483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a79900-0e5f-4bc3-a34c-0495dabdc1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.561759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a79900-0e5f-4bc3-a34c-0495dabdc1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.562500 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a79900-0e5f-4bc3-a34c-0495dabdc1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.571347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a79900-0e5f-4bc3-a34c-0495dabdc1d0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.587349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7e8c0-367b-4514-afbb-34f9933fd4b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.588821 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7e8c0-367b-4514-afbb-34f9933fd4b9 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.597250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 797/1586] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:09 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:09.597250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a79900-0e5f-4bc3-a34c-0495dabdc1d0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.599976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 790/1587] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:09.600728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7e8c0-367b-4514-afbb-34f9933fd4b9 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.604677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a79900-0e5f-4bc3-a34c-0495dabdc1d0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.609388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc6b73-d0d2-4455-8029-1abfd73a0dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.610113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc6b73-d0d2-4455-8029-1abfd73a0dbb admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.610402 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc6b73-d0d2-4455-8029-1abfd73a0dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.610688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc6b73-d0d2-4455-8029-1abfd73a0dbb admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.610805 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7e8c0-367b-4514-afbb-34f9933fd4b9 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.611461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc6b73-d0d2-4455-8029-1abfd73a0dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.613371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc6b73-d0d2-4455-8029-1abfd73a0dbb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.615598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2461d-ed05-4653-b57d-f8cca02fd5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.616378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2461d-ed05-4653-b57d-f8cca02fd5fe admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.616614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2461d-ed05-4653-b57d-f8cca02fd5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.616936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2461d-ed05-4653-b57d-f8cca02fd5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.617719 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2461d-ed05-4653-b57d-f8cca02fd5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.619477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2461d-ed05-4653-b57d-f8cca02fd5fe admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.623512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:09.637976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 791/1588] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.637976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2461d-ed05-4653-b57d-f8cca02fd5fe admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.643778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 798/1589] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:09 2022] POST /identity/v3/users => generated 285 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:09.643778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc6b73-d0d2-4455-8029-1abfd73a0dbb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.647352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2461d-ed05-4653-b57d-f8cca02fd5fe admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.654095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed207312-2057-4f56-8a8a-a72aa9fd3adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.655227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed207312-2057-4f56-8a8a-a72aa9fd3adb admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.655673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed207312-2057-4f56-8a8a-a72aa9fd3adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.655940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed207312-2057-4f56-8a8a-a72aa9fd3adb admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.656212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc6b73-d0d2-4455-8029-1abfd73a0dbb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.656679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed207312-2057-4f56-8a8a-a72aa9fd3adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.658392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed207312-2057-4f56-8a8a-a72aa9fd3adb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.663398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62124c56-1ce0-4703-aaa3-26d63404c601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.664105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62124c56-1ce0-4703-aaa3-26d63404c601 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.664504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62124c56-1ce0-4703-aaa3-26d63404c601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.665124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62124c56-1ce0-4703-aaa3-26d63404c601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.665996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62124c56-1ce0-4703-aaa3-26d63404c601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.667666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62124c56-1ce0-4703-aaa3-26d63404c601 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.678109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 792/1590] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.678109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed207312-2057-4f56-8a8a-a72aa9fd3adb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.688772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 799/1591] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.689156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed207312-2057-4f56-8a8a-a72aa9fd3adb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.693282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ec9c2-8ab9-435e-936b-03c5430bb32b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.693582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ec9c2-8ab9-435e-936b-03c5430bb32b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.693864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ec9c2-8ab9-435e-936b-03c5430bb32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.694544 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf817c-61c3-4f92-bf9c-152257676287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.696029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf817c-61c3-4f92-bf9c-152257676287 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.696029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf817c-61c3-4f92-bf9c-152257676287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.696381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf817c-61c3-4f92-bf9c-152257676287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.697187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf817c-61c3-4f92-bf9c-152257676287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.699219 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf817c-61c3-4f92-bf9c-152257676287 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.716014 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 793/1592] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.716014 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf817c-61c3-4f92-bf9c-152257676287 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.726868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf817c-61c3-4f92-bf9c-152257676287 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.731564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dd6ec9c2-8ab9-435e-936b-03c5430bb32b None None] MFA Rules not processed for user `baeb650373e84766a47f0825032a6648`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:09.732134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ee469-b838-4c58-a703-044ef423e12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.733038 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ee469-b838-4c58-a703-044ef423e12e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.733427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ee469-b838-4c58-a703-044ef423e12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.733551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dd6ec9c2-8ab9-435e-936b-03c5430bb32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:09.733698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ee469-b838-4c58-a703-044ef423e12e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.734557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ee469-b838-4c58-a703-044ef423e12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.736323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ee469-b838-4c58-a703-044ef423e12e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.749423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 794/1593] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.750331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ee469-b838-4c58-a703-044ef423e12e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.764321 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ee469-b838-4c58-a703-044ef423e12e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.774580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191e7df-ac3e-48c7-b9c9-f3f08e0e6551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.775267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191e7df-ac3e-48c7-b9c9-f3f08e0e6551 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.775692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191e7df-ac3e-48c7-b9c9-f3f08e0e6551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.775932 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191e7df-ac3e-48c7-b9c9-f3f08e0e6551 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.776872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191e7df-ac3e-48c7-b9c9-f3f08e0e6551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.778628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191e7df-ac3e-48c7-b9c9-f3f08e0e6551 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.786263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:09.796662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 795/1594] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.796662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191e7df-ac3e-48c7-b9c9-f3f08e0e6551 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.813312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191e7df-ac3e-48c7-b9c9-f3f08e0e6551 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.821567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88da7a7-a8fc-4032-842b-4ac97d27e992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.822417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88da7a7-a8fc-4032-842b-4ac97d27e992 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.822728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88da7a7-a8fc-4032-842b-4ac97d27e992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.823071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88da7a7-a8fc-4032-842b-4ac97d27e992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.823810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88da7a7-a8fc-4032-842b-4ac97d27e992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.826449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88da7a7-a8fc-4032-842b-4ac97d27e992 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.831456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 800/1595] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:09.835175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ec9c2-8ab9-435e-936b-03c5430bb32b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.844565 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 796/1596] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.845538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88da7a7-a8fc-4032-842b-4ac97d27e992 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.851245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ec9c2-8ab9-435e-936b-03c5430bb32b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.856254 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90138807-fb27-4d06-95d9-f65f4b9ea2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.857152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90138807-fb27-4d06-95d9-f65f4b9ea2f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.857443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90138807-fb27-4d06-95d9-f65f4b9ea2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.857724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90138807-fb27-4d06-95d9-f65f4b9ea2f4 admin admin] PATH_INFO: `/v3/projects/e8bedc0c93ca4c198b118bc51db92aa0/users/45e88807eb83454f8437224883b66c58/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.858181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88da7a7-a8fc-4032-842b-4ac97d27e992 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.863684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8c01b-a00a-4463-b45f-fd7565fbd625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.864437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8c01b-a00a-4463-b45f-fd7565fbd625 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.864791 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8c01b-a00a-4463-b45f-fd7565fbd625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.865140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8c01b-a00a-4463-b45f-fd7565fbd625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.866076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8c01b-a00a-4463-b45f-fd7565fbd625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.868183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8c01b-a00a-4463-b45f-fd7565fbd625 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.879658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 797/1597] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.880452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90138807-fb27-4d06-95d9-f65f4b9ea2f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8bedc0c93ca4c198b118bc51db92aa0, user_id=45e88807eb83454f8437224883b66c58, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.880592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8c01b-a00a-4463-b45f-fd7565fbd625 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.884089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90138807-fb27-4d06-95d9-f65f4b9ea2f4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.894181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8c01b-a00a-4463-b45f-fd7565fbd625 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.901705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 801/1598] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:09 2022] PUT /identity/v3/projects/e8bedc0c93ca4c198b118bc51db92aa0/users/45e88807eb83454f8437224883b66c58/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:09.903541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90138807-fb27-4d06-95d9-f65f4b9ea2f4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.965455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681442b-c0c0-4fd5-962f-976473aa9513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.966345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681442b-c0c0-4fd5-962f-976473aa9513 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.966742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681442b-c0c0-4fd5-962f-976473aa9513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.967085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681442b-c0c0-4fd5-962f-976473aa9513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.967910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681442b-c0c0-4fd5-962f-976473aa9513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.969651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681442b-c0c0-4fd5-962f-976473aa9513 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.969852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90138807-fb27-4d06-95d9-f65f4b9ea2f4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:09.976767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e28ab0-344d-4e9b-b10e-de7d14472424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:09.977466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e28ab0-344d-4e9b-b10e-de7d14472424 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:09.977746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e28ab0-344d-4e9b-b10e-de7d14472424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:09.978049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e28ab0-344d-4e9b-b10e-de7d14472424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:09.981983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e28ab0-344d-4e9b-b10e-de7d14472424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:09.982647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e28ab0-344d-4e9b-b10e-de7d14472424 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:09.983743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 798/1599] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.987117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681442b-c0c0-4fd5-962f-976473aa9513 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.997210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 802/1600] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:09.997210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e28ab0-344d-4e9b-b10e-de7d14472424 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:09.999466 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681442b-c0c0-4fd5-962f-976473aa9513 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:10.004566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbac5a-d7e6-4328-bccf-be7f38ee9ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:10.005535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbac5a-d7e6-4328-bccf-be7f38ee9ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.005828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbac5a-d7e6-4328-bccf-be7f38ee9ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.006165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbac5a-d7e6-4328-bccf-be7f38ee9ca5 admin admin] PATH_INFO: `/v3/projects/b2843a54fec340e19485e7b821fc071a/users/8561734cf36e47b5ab2d7242076437ff/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.023057 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbac5a-d7e6-4328-bccf-be7f38ee9ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2843a54fec340e19485e7b821fc071a, user_id=8561734cf36e47b5ab2d7242076437ff, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:10.025741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbac5a-d7e6-4328-bccf-be7f38ee9ca5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:10.048476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 799/1601] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:09 2022] PUT /identity/v3/projects/b2843a54fec340e19485e7b821fc071a/users/8561734cf36e47b5ab2d7242076437ff/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:10.049461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbac5a-d7e6-4328-bccf-be7f38ee9ca5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:10.075954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e28ab0-344d-4e9b-b10e-de7d14472424 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:10.141857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8c6a9-72af-4502-9b84-dddcd49d5a46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:10.142728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8c6a9-72af-4502-9b84-dddcd49d5a46 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.142998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8c6a9-72af-4502-9b84-dddcd49d5a46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.143284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8c6a9-72af-4502-9b84-dddcd49d5a46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.145932 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbac5a-d7e6-4328-bccf-be7f38ee9ca5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:10.151197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8d78e-e52b-46be-a346-25170b4dc2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:10.152047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8d78e-e52b-46be-a346-25170b4dc2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.152446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8d78e-e52b-46be-a346-25170b4dc2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.152733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8d78e-e52b-46be-a346-25170b4dc2e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.153482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8d78e-e52b-46be-a346-25170b4dc2e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:10.155395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8d78e-e52b-46be-a346-25170b4dc2e1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:10.158225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:10.161413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:10.177373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 800/1602] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:10.177373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8d78e-e52b-46be-a346-25170b4dc2e1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:10.191523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8d78e-e52b-46be-a346-25170b4dc2e1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:10.197533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae58de6-70b4-4aff-971d-d523cea5f524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:10.198903 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae58de6-70b4-4aff-971d-d523cea5f524 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.199403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae58de6-70b4-4aff-971d-d523cea5f524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.199689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae58de6-70b4-4aff-971d-d523cea5f524 admin admin] PATH_INFO: `/v3/projects/c743e22f3fdb4a61ad1b8035ac06d15b/users/dfeb642f25484665b7c477cde153162d/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.217617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae58de6-70b4-4aff-971d-d523cea5f524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c743e22f3fdb4a61ad1b8035ac06d15b, user_id=dfeb642f25484665b7c477cde153162d, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:10.218299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8c6a9-72af-4502-9b84-dddcd49d5a46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:10.219906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8c6a9-72af-4502-9b84-dddcd49d5a46 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:10.220970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae58de6-70b4-4aff-971d-d523cea5f524 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:10.231573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 803/1603] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:09 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:10.232740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879c292-dbc1-4fb2-83ba-3096a2cc6245 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.233202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879c292-dbc1-4fb2-83ba-3096a2cc6245 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.233544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879c292-dbc1-4fb2-83ba-3096a2cc6245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.250174 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 801/1604] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:10 2022] PUT /identity/v3/projects/c743e22f3fdb4a61ad1b8035ac06d15b/users/dfeb642f25484665b7c477cde153162d/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:10.250922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae58de6-70b4-4aff-971d-d523cea5f524 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:10.267943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2879c292-dbc1-4fb2-83ba-3096a2cc6245 None None] MFA Rules not processed for user `45e88807eb83454f8437224883b66c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:10.269159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2879c292-dbc1-4fb2-83ba-3096a2cc6245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:10.360322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 804/1605] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:10.361175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879c292-dbc1-4fb2-83ba-3096a2cc6245 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:10.364891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae58de6-70b4-4aff-971d-d523cea5f524 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:10.374496 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4407c-f6e3-4013-8559-45e19cec87d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:10.375582 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4407c-f6e3-4013-8559-45e19cec87d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.375973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4407c-f6e3-4013-8559-45e19cec87d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.376436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4407c-f6e3-4013-8559-45e19cec87d1 admin admin] PATH_INFO: `/v3/projects/2be6e5aedba545f4b562423c55ddbf80/users/15ec5e26839f494e98ec70ac49ebe415/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.377660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879c292-dbc1-4fb2-83ba-3096a2cc6245 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:10.383711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5342d-f985-42f7-be04-d2b803ea57a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:10.384801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5342d-f985-42f7-be04-d2b803ea57a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.385237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5342d-f985-42f7-be04-d2b803ea57a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.386226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5342d-f985-42f7-be04-d2b803ea57a3 admin admin] PATH_INFO: `/v3/projects/f1b4bcbfdd4a42e5a542dbf019166beb/users/5c54ac39faa84c7db5b9b63360216d7b/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.401692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4407c-f6e3-4013-8559-45e19cec87d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be6e5aedba545f4b562423c55ddbf80, user_id=15ec5e26839f494e98ec70ac49ebe415, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:10.406042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4407c-f6e3-4013-8559-45e19cec87d1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:10.411544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5342d-f985-42f7-be04-d2b803ea57a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b4bcbfdd4a42e5a542dbf019166beb, user_id=5c54ac39faa84c7db5b9b63360216d7b, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:10.414803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5342d-f985-42f7-be04-d2b803ea57a3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:10.432319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 805/1606] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:10 2022] PUT /identity/v3/projects/f1b4bcbfdd4a42e5a542dbf019166beb/users/5c54ac39faa84c7db5b9b63360216d7b/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:10.432549 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 802/1607] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:10 2022] PUT /identity/v3/projects/2be6e5aedba545f4b562423c55ddbf80/users/15ec5e26839f494e98ec70ac49ebe415/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:10.434399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5342d-f985-42f7-be04-d2b803ea57a3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:10.435250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dceec-f9f0-4ee5-b18f-0e8461586278 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.435738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dceec-f9f0-4ee5-b18f-0e8461586278 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.436149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dceec-f9f0-4ee5-b18f-0e8461586278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.471918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d47dceec-f9f0-4ee5-b18f-0e8461586278 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:10.473399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d47dceec-f9f0-4ee5-b18f-0e8461586278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:10.553080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5342d-f985-42f7-be04-d2b803ea57a3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:10.567023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c9f50-31bb-435c-904b-7ab06e3ef2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:10.567953 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c9f50-31bb-435c-904b-7ab06e3ef2d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.568232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c9f50-31bb-435c-904b-7ab06e3ef2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.569097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c9f50-31bb-435c-904b-7ab06e3ef2d6 admin admin] PATH_INFO: `/v3/projects/5d53da91e85a4a58af7b3b72563c03d2/users/df756f4e2c4e4a0fb13adc97f5affb1c/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.573505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 803/1608] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:10.576914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85699a-00c8-4416-9929-268793b454a7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.577236 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85699a-00c8-4416-9929-268793b454a7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.577487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85699a-00c8-4416-9929-268793b454a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.599031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c9f50-31bb-435c-904b-7ab06e3ef2d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d53da91e85a4a58af7b3b72563c03d2, user_id=df756f4e2c4e4a0fb13adc97f5affb1c, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:10.602234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c9f50-31bb-435c-904b-7ab06e3ef2d6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:10.614629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fe85699a-00c8-4416-9929-268793b454a7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:10.620659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fe85699a-00c8-4416-9929-268793b454a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:10.626461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 806/1609] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:10 2022] PUT /identity/v3/projects/5d53da91e85a4a58af7b3b72563c03d2/users/df756f4e2c4e4a0fb13adc97f5affb1c/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:10.638301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c9f50-31bb-435c-904b-7ab06e3ef2d6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:10.646467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:10.699713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:10.722299 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 804/1610] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:10.724527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b27c-9598-49f0-a775-e0ec920bc7f3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.724527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b27c-9598-49f0-a775-e0ec920bc7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.724527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b27c-9598-49f0-a775-e0ec920bc7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.750009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f902b27c-9598-49f0-a775-e0ec920bc7f3 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:10.751212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f902b27c-9598-49f0-a775-e0ec920bc7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:10.765922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 805/1611] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:10.767927 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec4b18-a147-4178-a263-b4e0f909a484 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.768544 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec4b18-a147-4178-a263-b4e0f909a484 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.768838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec4b18-a147-4178-a263-b4e0f909a484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.793300 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c9f50-31bb-435c-904b-7ab06e3ef2d6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:10.794138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f9ec4b18-a147-4178-a263-b4e0f909a484 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:10.795353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f9ec4b18-a147-4178-a263-b4e0f909a484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:10.799042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72954c10-dc69-41f6-8fa2-b31229f97ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:10.799941 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72954c10-dc69-41f6-8fa2-b31229f97ecc admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.800632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72954c10-dc69-41f6-8fa2-b31229f97ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.801108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72954c10-dc69-41f6-8fa2-b31229f97ecc admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.802160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72954c10-dc69-41f6-8fa2-b31229f97ecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:10.804578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72954c10-dc69-41f6-8fa2-b31229f97ecc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:10.808897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 806/1612] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:10.809856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec4b18-a147-4178-a263-b4e0f909a484 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:10.846349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 807/1613] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:10 2022] POST /identity/v3/projects => generated 357 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:10.848256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef07d88-9a0c-43e5-bf7f-24cefd56446f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.848909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef07d88-9a0c-43e5-bf7f-24cefd56446f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.849622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef07d88-9a0c-43e5-bf7f-24cefd56446f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:10.883058 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3ef07d88-9a0c-43e5-bf7f-24cefd56446f None None] MFA Rules not processed for user `df756f4e2c4e4a0fb13adc97f5affb1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:10.884579 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3ef07d88-9a0c-43e5-bf7f-24cefd56446f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:10.892667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec4b18-a147-4178-a263-b4e0f909a484 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:10.983917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dfd35-b6f2-4e2c-ab75-81e57973801c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:10.984593 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dfd35-b6f2-4e2c-ab75-81e57973801c service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:10.985045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dfd35-b6f2-4e2c-ab75-81e57973801c service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:10.985384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dfd35-b6f2-4e2c-ab75-81e57973801c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.006490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 808/1614] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:11.007747 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef07d88-9a0c-43e5-bf7f-24cefd56446f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.022078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef07d88-9a0c-43e5-bf7f-24cefd56446f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.027667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64edda8a-be80-48df-9326-c326ce656451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:11.028413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64edda8a-be80-48df-9326-c326ce656451 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:11.028849 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64edda8a-be80-48df-9326-c326ce656451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:11.029069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64edda8a-be80-48df-9326-c326ce656451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.086238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2dfd35-b6f2-4e2c-ab75-81e57973801c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.087706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2dfd35-b6f2-4e2c-ab75-81e57973801c service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:11.105800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 807/1615] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:11.106862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dfd35-b6f2-4e2c-ab75-81e57973801c service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.110471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4a2dfd35-b6f2-4e2c-ab75-81e57973801c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:11.125796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dfd35-b6f2-4e2c-ab75-81e57973801c service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.145994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5f4c9-8305-43de-a0b2-d053baffae39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:11.145994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5f4c9-8305-43de-a0b2-d053baffae39 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:11.145994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5f4c9-8305-43de-a0b2-d053baffae39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:11.145994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5f4c9-8305-43de-a0b2-d053baffae39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.148711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64edda8a-be80-48df-9326-c326ce656451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.150297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64edda8a-be80-48df-9326-c326ce656451 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:11.165410 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 809/1616] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:11.166604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64edda8a-be80-48df-9326-c326ce656451 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.179017 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64edda8a-be80-48df-9326-c326ce656451 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.181437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5f4c9-8305-43de-a0b2-d053baffae39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.182366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5f4c9-8305-43de-a0b2-d053baffae39 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:11.184115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad297d-c3b2-4c54-8d8b-bfa98c363877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:11.184912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad297d-c3b2-4c54-8d8b-bfa98c363877 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:11.185296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad297d-c3b2-4c54-8d8b-bfa98c363877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:11.185670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad297d-c3b2-4c54-8d8b-bfa98c363877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.198620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad297d-c3b2-4c54-8d8b-bfa98c363877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.200643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad297d-c3b2-4c54-8d8b-bfa98c363877 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:11.208522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:11.216478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 810/1617] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:11.218222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad297d-c3b2-4c54-8d8b-bfa98c363877 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.235230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 808/1618] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:11.236100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5f4c9-8305-43de-a0b2-d053baffae39 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.243948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad297d-c3b2-4c54-8d8b-bfa98c363877 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.255743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6d06e-fe5c-4bfc-ac03-9f96ecefeb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:11.256086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6d06e-fe5c-4bfc-ac03-9f96ecefeb5e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:11.257168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6d06e-fe5c-4bfc-ac03-9f96ecefeb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:11.257552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6d06e-fe5c-4bfc-ac03-9f96ecefeb5e admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.258369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6d06e-fe5c-4bfc-ac03-9f96ecefeb5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.262142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6d06e-fe5c-4bfc-ac03-9f96ecefeb5e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:11.263189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5f4c9-8305-43de-a0b2-d053baffae39 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.278029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2f360-edf4-4e4d-9147-8ec4e397ca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:11.279106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2f360-edf4-4e4d-9147-8ec4e397ca1b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:11.279553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2f360-edf4-4e4d-9147-8ec4e397ca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:11.280246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2f360-edf4-4e4d-9147-8ec4e397ca1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.282027 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2f360-edf4-4e4d-9147-8ec4e397ca1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.288396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2f360-edf4-4e4d-9147-8ec4e397ca1b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:11.312252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 811/1619] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:11 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:11.313641 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6d06e-fe5c-4bfc-ac03-9f96ecefeb5e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.330350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6d06e-fe5c-4bfc-ac03-9f96ecefeb5e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.337447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fa2eb-9733-4abc-a824-412caf8243a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:11.344167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa2eb-9733-4abc-a824-412caf8243a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:11.344467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa2eb-9733-4abc-a824-412caf8243a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:11.344750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa2eb-9733-4abc-a824-412caf8243a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.345514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fa2eb-9733-4abc-a824-412caf8243a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.348864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fa2eb-9733-4abc-a824-412caf8243a9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:11.358592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 809/1620] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:11 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:11.359603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2f360-edf4-4e4d-9147-8ec4e397ca1b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.380265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 812/1621] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:11.381342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fa2eb-9733-4abc-a824-412caf8243a9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.387689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:11.405790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fa2eb-9733-4abc-a824-412caf8243a9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.484114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2f360-edf4-4e4d-9147-8ec4e397ca1b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.491356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab008958-67d5-4c9f-a53b-30410b9b5334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:11.492148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab008958-67d5-4c9f-a53b-30410b9b5334 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:11.492494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab008958-67d5-4c9f-a53b-30410b9b5334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:11.492809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab008958-67d5-4c9f-a53b-30410b9b5334 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.493933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab008958-67d5-4c9f-a53b-30410b9b5334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.496914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab008958-67d5-4c9f-a53b-30410b9b5334 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:11.528892 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 810/1622] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:11 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:30:11.532845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab008958-67d5-4c9f-a53b-30410b9b5334 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.542585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec005268-1d0c-4778-a19f-b962906bb2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:11.543883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec005268-1d0c-4778-a19f-b962906bb2b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:11.544333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec005268-1d0c-4778-a19f-b962906bb2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:11.544760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec005268-1d0c-4778-a19f-b962906bb2b3 admin admin] PATH_INFO: `/v3/projects/5f3d8cf6e6734049b474c2e1b60dd232/users/29d74757a46b4b4c8dad8a0fb6a83d38/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.553414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab008958-67d5-4c9f-a53b-30410b9b5334 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.563385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae821fdb-dd3b-4634-a464-5db293f92d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:11.564321 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae821fdb-dd3b-4634-a464-5db293f92d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:11.564831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae821fdb-dd3b-4634-a464-5db293f92d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:11.565759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae821fdb-dd3b-4634-a464-5db293f92d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.567505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae821fdb-dd3b-4634-a464-5db293f92d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.575908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae821fdb-dd3b-4634-a464-5db293f92d7c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:11.592383 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec005268-1d0c-4778-a19f-b962906bb2b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3d8cf6e6734049b474c2e1b60dd232, user_id=29d74757a46b4b4c8dad8a0fb6a83d38, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.595868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec005268-1d0c-4778-a19f-b962906bb2b3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:11.611162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 811/1623] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:30:11.612362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae821fdb-dd3b-4634-a464-5db293f92d7c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.629835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 813/1624] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:11 2022] PUT /identity/v3/projects/5f3d8cf6e6734049b474c2e1b60dd232/users/29d74757a46b4b4c8dad8a0fb6a83d38/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:11.635772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec005268-1d0c-4778-a19f-b962906bb2b3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.639967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae821fdb-dd3b-4634-a464-5db293f92d7c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.747581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da767a-f852-44bd-8aea-ab7f1f893491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:11.748502 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da767a-f852-44bd-8aea-ab7f1f893491 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:11.748793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da767a-f852-44bd-8aea-ab7f1f893491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:11.749074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da767a-f852-44bd-8aea-ab7f1f893491 admin admin] PATH_INFO: `/v3/projects/d53a6b8ca68d41758e762294eecc0a1b/users/3974efc95d5d46fa9e4bf4ae1e35a692/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.774960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da767a-f852-44bd-8aea-ab7f1f893491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53a6b8ca68d41758e762294eecc0a1b, user_id=3974efc95d5d46fa9e4bf4ae1e35a692, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.779165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da767a-f852-44bd-8aea-ab7f1f893491 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:11.804879 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 812/1625] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:11 2022] PUT /identity/v3/projects/d53a6b8ca68d41758e762294eecc0a1b/users/3974efc95d5d46fa9e4bf4ae1e35a692/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:11.806318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da767a-f852-44bd-8aea-ab7f1f893491 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:11.880248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec005268-1d0c-4778-a19f-b962906bb2b3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.940345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da767a-f852-44bd-8aea-ab7f1f893491 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:11.945885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc9cef-9aa9-4e9a-a7fd-052c3be90e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:11.946735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc9cef-9aa9-4e9a-a7fd-052c3be90e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:11.947024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc9cef-9aa9-4e9a-a7fd-052c3be90e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:11.947303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc9cef-9aa9-4e9a-a7fd-052c3be90e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:11.948046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc9cef-9aa9-4e9a-a7fd-052c3be90e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:11.965101 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc9cef-9aa9-4e9a-a7fd-052c3be90e4b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:12.007946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 813/1626] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:30:12.011347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc9cef-9aa9-4e9a-a7fd-052c3be90e4b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:12.060890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc9cef-9aa9-4e9a-a7fd-052c3be90e4b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:12.071806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc026120-c810-4635-9737-f32cb8571a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:12.072857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc026120-c810-4635-9737-f32cb8571a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:12.073553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc026120-c810-4635-9737-f32cb8571a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:12.073553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc026120-c810-4635-9737-f32cb8571a56 admin admin] PATH_INFO: `/v3/projects/d53a6b8ca68d41758e762294eecc0a1b/users/3974efc95d5d46fa9e4bf4ae1e35a692/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:12.076346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5591f-77e6-497d-8459-5140ea1fe41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:12.080717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5591f-77e6-497d-8459-5140ea1fe41c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:12.080977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5591f-77e6-497d-8459-5140ea1fe41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:12.082167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5591f-77e6-497d-8459-5140ea1fe41c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:12.082875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5591f-77e6-497d-8459-5140ea1fe41c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:12.084702 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc026120-c810-4635-9737-f32cb8571a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53a6b8ca68d41758e762294eecc0a1b, user_id=3974efc95d5d46fa9e4bf4ae1e35a692, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:12.087536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc026120-c810-4635-9737-f32cb8571a56 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:12.088415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5591f-77e6-497d-8459-5140ea1fe41c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:12.093207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:12.106188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:12.120663 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 814/1627] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:12.124398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5591f-77e6-497d-8459-5140ea1fe41c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:12.125064 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 814/1628] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:12 2022] PUT /identity/v3/projects/d53a6b8ca68d41758e762294eecc0a1b/users/3974efc95d5d46fa9e4bf4ae1e35a692/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:30:12.142354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc775-3125-45bf-8103-1405a83c57d9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:12.142721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc775-3125-45bf-8103-1405a83c57d9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:12.144665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc775-3125-45bf-8103-1405a83c57d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:12.160172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5591f-77e6-497d-8459-5140ea1fe41c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:12.208279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-68cfc775-3125-45bf-8103-1405a83c57d9 None None] MFA Rules not processed for user `3974efc95d5d46fa9e4bf4ae1e35a692`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:12.209405 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-68cfc775-3125-45bf-8103-1405a83c57d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:12.391660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 815/1629] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:12.406554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfc775-3125-45bf-8103-1405a83c57d9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:12.515334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b42f5-e1ad-4790-8c3b-95ef4b7868ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:12.526086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b42f5-e1ad-4790-8c3b-95ef4b7868ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:12.526402 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b42f5-e1ad-4790-8c3b-95ef4b7868ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:12.526668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b42f5-e1ad-4790-8c3b-95ef4b7868ee admin admin] PATH_INFO: `/v3/projects/5f3d8cf6e6734049b474c2e1b60dd232/users/29d74757a46b4b4c8dad8a0fb6a83d38/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:12.530615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b42f5-e1ad-4790-8c3b-95ef4b7868ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3d8cf6e6734049b474c2e1b60dd232, user_id=29d74757a46b4b4c8dad8a0fb6a83d38, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:12.536860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b42f5-e1ad-4790-8c3b-95ef4b7868ee admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:12.571490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 815/1630] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:12 2022] PUT /identity/v3/projects/5f3d8cf6e6734049b474c2e1b60dd232/users/29d74757a46b4b4c8dad8a0fb6a83d38/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:12.584219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860386e3-dcd3-497d-acdc-1fbcb7961d8a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:12.584413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860386e3-dcd3-497d-acdc-1fbcb7961d8a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:12.584699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860386e3-dcd3-497d-acdc-1fbcb7961d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:12.603243 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfc775-3125-45bf-8103-1405a83c57d9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:12.678446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-860386e3-dcd3-497d-acdc-1fbcb7961d8a None None] MFA Rules not processed for user `29d74757a46b4b4c8dad8a0fb6a83d38`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:12.679679 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-860386e3-dcd3-497d-acdc-1fbcb7961d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:12.779244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f546a1-24de-41c1-81c6-cf1023fe650e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:12.780187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f546a1-24de-41c1-81c6-cf1023fe650e service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:12.780479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f546a1-24de-41c1-81c6-cf1023fe650e service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:12.780750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f546a1-24de-41c1-81c6-cf1023fe650e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:12.904050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:12.926622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 816/1631] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:12 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:12.943529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860386e3-dcd3-497d-acdc-1fbcb7961d8a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:12.951308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f546a1-24de-41c1-81c6-cf1023fe650e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:12.955369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f546a1-24de-41c1-81c6-cf1023fe650e service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:12.963316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860386e3-dcd3-497d-acdc-1fbcb7961d8a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:12.973483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87548ec8-89bf-40a7-80d7-56fec83e1720 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:12.975407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87548ec8-89bf-40a7-80d7-56fec83e1720 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:12.976877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87548ec8-89bf-40a7-80d7-56fec83e1720 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:12.977310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87548ec8-89bf-40a7-80d7-56fec83e1720 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:12.979960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 816/1632] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:12.994783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87548ec8-89bf-40a7-80d7-56fec83e1720 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:12.996621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87548ec8-89bf-40a7-80d7-56fec83e1720 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:13.021220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 817/1633] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:12 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:13.129357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f546a1-24de-41c1-81c6-cf1023fe650e service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:13.141009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f546a1-24de-41c1-81c6-cf1023fe650e service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:13.147972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec043eb-db6d-4418-9b0a-d9b8392831e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:13.149097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec043eb-db6d-4418-9b0a-d9b8392831e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:13.149645 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec043eb-db6d-4418-9b0a-d9b8392831e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:13.150026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec043eb-db6d-4418-9b0a-d9b8392831e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:13.198649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e140eb4-6539-44b9-8789-9c10809250be None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:13.198935 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e140eb4-6539-44b9-8789-9c10809250be None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:13.199331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e140eb4-6539-44b9-8789-9c10809250be None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:13.202037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 818/1634] 10.222.0.42 () {54 vars in 962 bytes} [Thu Jul 21 06:30:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:30:13.214300 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ba700-36f9-4214-96a0-3356c57e8895 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:13.214937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ba700-36f9-4214-96a0-3356c57e8895 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:13.215281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ba700-36f9-4214-96a0-3356c57e8895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:13.380334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e26ba700-36f9-4214-96a0-3356c57e8895 None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:13.387458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e26ba700-36f9-4214-96a0-3356c57e8895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:13.415288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec043eb-db6d-4418-9b0a-d9b8392831e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:13.416742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec043eb-db6d-4418-9b0a-d9b8392831e7 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:13.468162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 817/1635] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:13 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:13.574533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:13.624832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 819/1636] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Jul 21 06:30:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:13.631810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fb763-5c51-4bc2-a267-1bb199e69899 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:13.632151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fb763-5c51-4bc2-a267-1bb199e69899 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:13.632664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fb763-5c51-4bc2-a267-1bb199e69899 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:13.634041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 818/1637] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jul 21 06:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:30:13.645223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a457dcb-7b77-4451-a13d-ae4e9911b0f6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:13.646362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a457dcb-7b77-4451-a13d-ae4e9911b0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:13.647167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a457dcb-7b77-4451-a13d-ae4e9911b0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:13.700253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1a457dcb-7b77-4451-a13d-ae4e9911b0f6 None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:13.703543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1a457dcb-7b77-4451-a13d-ae4e9911b0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:13.753051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 820/1638] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:30:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:13.828643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fb763-5c51-4bc2-a267-1bb199e69899 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:13.859524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fb763-5c51-4bc2-a267-1bb199e69899 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:13.867079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f80a32-c1bc-447c-aba6-ba17ca88610f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:13.867971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f80a32-c1bc-447c-aba6-ba17ca88610f service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:13.868267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f80a32-c1bc-447c-aba6-ba17ca88610f service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:13.868585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f80a32-c1bc-447c-aba6-ba17ca88610f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:13.885579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f80a32-c1bc-447c-aba6-ba17ca88610f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:13.888007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f80a32-c1bc-447c-aba6-ba17ca88610f service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:13.914762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 819/1639] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:30:16.307218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f008d1-6566-42aa-a4bd-bcece1e2b2ba None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:16.307790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f008d1-6566-42aa-a4bd-bcece1e2b2ba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:16.308489 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f008d1-6566-42aa-a4bd-bcece1e2b2ba None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:16.310504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 821/1640] 10.222.0.42 () {54 vars in 962 bytes} [Thu Jul 21 06:30:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:30:16.327678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329e851-9b48-4e18-a18a-db77dd195425 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:16.328190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329e851-9b48-4e18-a18a-db77dd195425 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:16.328601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329e851-9b48-4e18-a18a-db77dd195425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:16.383166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a329e851-9b48-4e18-a18a-db77dd195425 None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:16.390382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a329e851-9b48-4e18-a18a-db77dd195425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:16.445818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 820/1641] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Jul 21 06:30:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:16.452565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988eea5-c49e-4a2d-9eb6-dd2b3c4b569a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:16.452933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988eea5-c49e-4a2d-9eb6-dd2b3c4b569a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:16.453246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988eea5-c49e-4a2d-9eb6-dd2b3c4b569a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:16.456810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 822/1642] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jul 21 06:30:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:30:16.461727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02f64f-1130-4f5e-8479-18eb52f9940f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:16.462243 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02f64f-1130-4f5e-8479-18eb52f9940f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:16.462683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02f64f-1130-4f5e-8479-18eb52f9940f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:16.546316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4c02f64f-1130-4f5e-8479-18eb52f9940f None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:16.557673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4c02f64f-1130-4f5e-8479-18eb52f9940f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:16.598197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 821/1643] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:30:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:16.954434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988eea5-c49e-4a2d-9eb6-dd2b3c4b569a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:16.975640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988eea5-c49e-4a2d-9eb6-dd2b3c4b569a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:16.997101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea1883-4620-4cb7-8c62-af09e88e4973 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:16.999020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1883-4620-4cb7-8c62-af09e88e4973 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:16.999513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1883-4620-4cb7-8c62-af09e88e4973 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:17.000306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1883-4620-4cb7-8c62-af09e88e4973 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:17.023795 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea1883-4620-4cb7-8c62-af09e88e4973 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:17.035052 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea1883-4620-4cb7-8c62-af09e88e4973 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:17.086766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 823/1644] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:17.112449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02f64f-1130-4f5e-8479-18eb52f9940f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:17.293224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02f64f-1130-4f5e-8479-18eb52f9940f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:17.304366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:17.305277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:17.305562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:17.308069 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] PATH_INFO: `/v3/users/df756f4e2c4e4a0fb13adc97f5affb1c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:17.312969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df756f4e2c4e4a0fb13adc97f5affb1c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:17.315739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:17.434525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df756f4e2c4e4a0fb13adc97f5affb1c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:17.465205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df756f4e2c4e4a0fb13adc97f5affb1c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:17.466958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:17.485672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df756f4e2c4e4a0fb13adc97f5affb1c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:17.487830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df756f4e2c4e4a0fb13adc97f5affb1c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:17.489966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df756f4e2c4e4a0fb13adc97f5affb1c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:17.521840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 822/1645] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:30:17 2022] DELETE /identity/v3/users/df756f4e2c4e4a0fb13adc97f5affb1c => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:17.527205 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e67815-869b-4bbb-9cf6-ddc2943d3f9e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:17.528051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e67815-869b-4bbb-9cf6-ddc2943d3f9e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:17.528542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e67815-869b-4bbb-9cf6-ddc2943d3f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:17.567984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-68e67815-869b-4bbb-9cf6-ddc2943d3f9e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:17.575958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-68e67815-869b-4bbb-9cf6-ddc2943d3f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:17.748255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 824/1646] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:17.779190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:17.781188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:17.887279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90552ae7-df85-4183-b688-9cd15decb246 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:17.892037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a3195-e5fb-4715-8f39-8a1bd2852e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:17.893296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a3195-e5fb-4715-8f39-8a1bd2852e01 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:17.893726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a3195-e5fb-4715-8f39-8a1bd2852e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:17.894345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a3195-e5fb-4715-8f39-8a1bd2852e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:17.907718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a3195-e5fb-4715-8f39-8a1bd2852e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:17.909547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a3195-e5fb-4715-8f39-8a1bd2852e01 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:17.983332 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 823/1647] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:19.244742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e67815-869b-4bbb-9cf6-ddc2943d3f9e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:19.249595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-68e67815-869b-4bbb-9cf6-ddc2943d3f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:19.269137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e67815-869b-4bbb-9cf6-ddc2943d3f9e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:19.291013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:19.293601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:19.300509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:19.301021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] PATH_INFO: `/v3/projects/5d53da91e85a4a58af7b3b72563c03d2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:19.305079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d53da91e85a4a58af7b3b72563c03d2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:19.319282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:19.453595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d53da91e85a4a58af7b3b72563c03d2', 'request_id': 'req-ccf78eeb-3dea-432c-b8ad-6247cad12818'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:19.469805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d53da91e85a4a58af7b3b72563c03d2', 'request_id': 'req-ccf78eeb-3dea-432c-b8ad-6247cad12818'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:19.496242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d53da91e85a4a58af7b3b72563c03d2', 'request_id': 'req-ccf78eeb-3dea-432c-b8ad-6247cad12818'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:19.522589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] The token cache is being invalidate because project 5d53da91e85a4a58af7b3b72563c03d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:30:19.526248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:19.528245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:19.533562 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 825/1648] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:30:19 2022] DELETE /identity/v3/projects/5d53da91e85a4a58af7b3b72563c03d2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:19.537583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a3195-e5fb-4715-8f39-8a1bd2852e01 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:19.558046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a3195-e5fb-4715-8f39-8a1bd2852e01 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:19.656092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87910fe4-6d26-4ee6-89fc-a7c149fb596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:19.657347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87910fe4-6d26-4ee6-89fc-a7c149fb596b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:19.657733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87910fe4-6d26-4ee6-89fc-a7c149fb596b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:19.662254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87910fe4-6d26-4ee6-89fc-a7c149fb596b admin admin] PATH_INFO: `/v3/users/3974efc95d5d46fa9e4bf4ae1e35a692` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:19.666983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87910fe4-6d26-4ee6-89fc-a7c149fb596b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3974efc95d5d46fa9e4bf4ae1e35a692)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:19.670512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87910fe4-6d26-4ee6-89fc-a7c149fb596b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:19.786282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-87910fe4-6d26-4ee6-89fc-a7c149fb596b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3974efc95d5d46fa9e4bf4ae1e35a692'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:19.831730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-87910fe4-6d26-4ee6-89fc-a7c149fb596b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3974efc95d5d46fa9e4bf4ae1e35a692'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:19.855761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-87910fe4-6d26-4ee6-89fc-a7c149fb596b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3974efc95d5d46fa9e4bf4ae1e35a692'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:19.857818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-87910fe4-6d26-4ee6-89fc-a7c149fb596b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3974efc95d5d46fa9e4bf4ae1e35a692'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:19.859536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-87910fe4-6d26-4ee6-89fc-a7c149fb596b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3974efc95d5d46fa9e4bf4ae1e35a692'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:19.915364 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 824/1649] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:30:19 2022] DELETE /identity/v3/users/3974efc95d5d46fa9e4bf4ae1e35a692 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:21.938339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:21.939808 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:21.978231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf78eeb-3dea-432c-b8ad-6247cad12818 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:22.185848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:22.217703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:22.218725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:22.219085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:22.220382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] PATH_INFO: `/v3/projects/d53a6b8ca68d41758e762294eecc0a1b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:22.227056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d53a6b8ca68d41758e762294eecc0a1b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:22.229158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:22.386942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd53a6b8ca68d41758e762294eecc0a1b', 'request_id': 'req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:22.406821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd53a6b8ca68d41758e762294eecc0a1b', 'request_id': 'req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:22.424457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd53a6b8ca68d41758e762294eecc0a1b', 'request_id': 'req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:22.458902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] The token cache is being invalidate because project d53a6b8ca68d41758e762294eecc0a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:30:22.459191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:22.460393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc7674ec-0fa0-42c6-992f-e9585be0ca1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:22.462730 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 826/1650] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:30:21 2022] DELETE /identity/v3/projects/d53a6b8ca68d41758e762294eecc0a1b => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:22.468163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b929872-0e80-4021-a000-fac1ccd705b6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:22.469816 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b929872-0e80-4021-a000-fac1ccd705b6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:22.470318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b929872-0e80-4021-a000-fac1ccd705b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:22.493050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:22.535644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9b929872-0e80-4021-a000-fac1ccd705b6 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:22.544081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9b929872-0e80-4021-a000-fac1ccd705b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:22.697516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d88181-9077-4cb6-ae8d-79037cd494f0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:22.697814 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d88181-9077-4cb6-ae8d-79037cd494f0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:22.698846 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d88181-9077-4cb6-ae8d-79037cd494f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:22.802721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 825/1651] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:22.804270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dfb5d-822e-46b8-a65d-361e891c0364 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:22.804563 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dfb5d-822e-46b8-a65d-361e891c0364 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:22.804852 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dfb5d-822e-46b8-a65d-361e891c0364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:22.821933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f4d88181-9077-4cb6-ae8d-79037cd494f0 None None] MFA Rules not processed for user `774dd66c674a4955a439afd30354efab`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:22.878798 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f4d88181-9077-4cb6-ae8d-79037cd494f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:22.917458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-af3dfb5d-822e-46b8-a65d-361e891c0364 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:22.918994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-af3dfb5d-822e-46b8-a65d-361e891c0364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:22.980547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 826/1652] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:22.985497 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dfb5d-822e-46b8-a65d-361e891c0364 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:23.003689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dfb5d-822e-46b8-a65d-361e891c0364 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:23.016439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555fbf0-5e47-4ea7-949b-d7e1c09790ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:23.017152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555fbf0-5e47-4ea7-949b-d7e1c09790ad admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:23.017436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555fbf0-5e47-4ea7-949b-d7e1c09790ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:23.017721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555fbf0-5e47-4ea7-949b-d7e1c09790ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:23.018511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555fbf0-5e47-4ea7-949b-d7e1c09790ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:23.018824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555fbf0-5e47-4ea7-949b-d7e1c09790ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:23.028358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555fbf0-5e47-4ea7-949b-d7e1c09790ad admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:23.099942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 827/1653] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:23.100901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555fbf0-5e47-4ea7-949b-d7e1c09790ad admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:23.112699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 827/1654] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:23.124523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76038737-0df9-451c-8c5c-1d1c75b05502 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:23.124893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76038737-0df9-451c-8c5c-1d1c75b05502 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:23.125257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76038737-0df9-451c-8c5c-1d1c75b05502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:23.147375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555fbf0-5e47-4ea7-949b-d7e1c09790ad admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:23.162065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a59b0-b046-48ef-94a0-e9e55bb01c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:23.162769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a59b0-b046-48ef-94a0-e9e55bb01c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:23.163059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a59b0-b046-48ef-94a0-e9e55bb01c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:23.163343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a59b0-b046-48ef-94a0-e9e55bb01c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:23.163605 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-76038737-0df9-451c-8c5c-1d1c75b05502 None None] MFA Rules not processed for user `774dd66c674a4955a439afd30354efab`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:23.164054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a59b0-b046-48ef-94a0-e9e55bb01c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:23.164979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-76038737-0df9-451c-8c5c-1d1c75b05502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:23.168142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a59b0-b046-48ef-94a0-e9e55bb01c66 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:23.187335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 828/1655] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:23.202376 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76038737-0df9-451c-8c5c-1d1c75b05502 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:23.204390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-76038737-0df9-451c-8c5c-1d1c75b05502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:23.243427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 828/1656] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:23 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:23.244339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a59b0-b046-48ef-94a0-e9e55bb01c66 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:23.354067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:23.378229 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a59b0-b046-48ef-94a0-e9e55bb01c66 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:23.390546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590680-8efa-4856-8ffe-8a0600138ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:23.391355 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590680-8efa-4856-8ffe-8a0600138ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:23.391533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590680-8efa-4856-8ffe-8a0600138ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:23.391817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590680-8efa-4856-8ffe-8a0600138ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:23.392528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590680-8efa-4856-8ffe-8a0600138ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:23.393583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76038737-0df9-451c-8c5c-1d1c75b05502 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:23.394547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590680-8efa-4856-8ffe-8a0600138ab3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:23.442905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 829/1657] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:23 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:23.447459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590680-8efa-4856-8ffe-8a0600138ab3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:23.475374 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590680-8efa-4856-8ffe-8a0600138ab3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:23.481530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e0c52-85fe-4ecd-af13-dc6b4916aa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:23.482452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e0c52-85fe-4ecd-af13-dc6b4916aa44 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:23.482896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e0c52-85fe-4ecd-af13-dc6b4916aa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:23.483314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e0c52-85fe-4ecd-af13-dc6b4916aa44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:23.484205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e0c52-85fe-4ecd-af13-dc6b4916aa44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:23.486004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e0c52-85fe-4ecd-af13-dc6b4916aa44 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:23.509378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 830/1658] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:23.513281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e0c52-85fe-4ecd-af13-dc6b4916aa44 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:23.536488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e0c52-85fe-4ecd-af13-dc6b4916aa44 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:23.560102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fbfbb-dded-4bf4-8a2e-0116b938da6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:23.561062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fbfbb-dded-4bf4-8a2e-0116b938da6e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:23.561499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fbfbb-dded-4bf4-8a2e-0116b938da6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:23.561915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fbfbb-dded-4bf4-8a2e-0116b938da6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:23.562841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fbfbb-dded-4bf4-8a2e-0116b938da6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:23.564601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fbfbb-dded-4bf4-8a2e-0116b938da6e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:23.576918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436b954-3e7a-4148-bd4e-b09e54a79b49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:23.584529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436b954-3e7a-4148-bd4e-b09e54a79b49 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:23.584529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436b954-3e7a-4148-bd4e-b09e54a79b49 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:23.584529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436b954-3e7a-4148-bd4e-b09e54a79b49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:23.595731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 831/1659] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:23.602430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fbfbb-dded-4bf4-8a2e-0116b938da6e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:23.632455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fbfbb-dded-4bf4-8a2e-0116b938da6e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:23.643223 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf645705-8b2f-4a52-9a8c-37da68e8b7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:23.644422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf645705-8b2f-4a52-9a8c-37da68e8b7bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:23.644843 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf645705-8b2f-4a52-9a8c-37da68e8b7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:23.645276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf645705-8b2f-4a52-9a8c-37da68e8b7bf admin admin] PATH_INFO: `/v3/projects/3fb45bcd0dfb4f2cbe8abd3af76b0a5e/users/b8d4d9874077426297e6eb717d558f45/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:23.684988 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf645705-8b2f-4a52-9a8c-37da68e8b7bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb45bcd0dfb4f2cbe8abd3af76b0a5e, user_id=b8d4d9874077426297e6eb717d558f45, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:23.690923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf645705-8b2f-4a52-9a8c-37da68e8b7bf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:23.729780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 832/1660] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:23 2022] PUT /identity/v3/projects/3fb45bcd0dfb4f2cbe8abd3af76b0a5e/users/b8d4d9874077426297e6eb717d558f45/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:23.741358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436b954-3e7a-4148-bd4e-b09e54a79b49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:23.741572 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2abcc-6b7e-4818-8c10-5294c8ea0961 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:23.741572 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2abcc-6b7e-4818-8c10-5294c8ea0961 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:23.741572 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2abcc-6b7e-4818-8c10-5294c8ea0961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:23.756504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436b954-3e7a-4148-bd4e-b09e54a79b49 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:23.774094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:23.787974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 829/1661] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:23.801738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cda2abcc-6b7e-4818-8c10-5294c8ea0961 None None] MFA Rules not processed for user `b8d4d9874077426297e6eb717d558f45`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:23.803014 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cda2abcc-6b7e-4818-8c10-5294c8ea0961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:23.902008 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 833/1662] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:23.904493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436b954-3e7a-4148-bd4e-b09e54a79b49 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:23.944464 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436b954-3e7a-4148-bd4e-b09e54a79b49 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:24.159841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb543d5-1d10-4560-b816-433f36449587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:24.160600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb543d5-1d10-4560-b816-433f36449587 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:24.160830 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb543d5-1d10-4560-b816-433f36449587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:24.161114 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb543d5-1d10-4560-b816-433f36449587 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:24.161827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb543d5-1d10-4560-b816-433f36449587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:24.164110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb543d5-1d10-4560-b816-433f36449587 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:24.242369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 830/1663] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:23 2022] POST /identity/v3/projects => generated 375 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:24.246971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2abcc-6b7e-4818-8c10-5294c8ea0961 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:24.267288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2abcc-6b7e-4818-8c10-5294c8ea0961 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:24.450835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b28987-7161-4f4b-bb3b-14b18edc0d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:24.451596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b28987-7161-4f4b-bb3b-14b18edc0d29 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:24.451941 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b28987-7161-4f4b-bb3b-14b18edc0d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:24.452297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b28987-7161-4f4b-bb3b-14b18edc0d29 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:24.452901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b28987-7161-4f4b-bb3b-14b18edc0d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:24.454981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b28987-7161-4f4b-bb3b-14b18edc0d29 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:24.503239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 834/1664] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:24 2022] POST /identity/v3/users => generated 294 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:24.504791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb543d5-1d10-4560-b816-433f36449587 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:24.520561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb543d5-1d10-4560-b816-433f36449587 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:24.531292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf87e28-a1de-4b1c-a232-500167ab2537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:24.531626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf87e28-a1de-4b1c-a232-500167ab2537 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:24.531902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf87e28-a1de-4b1c-a232-500167ab2537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:24.532186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf87e28-a1de-4b1c-a232-500167ab2537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:24.532933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf87e28-a1de-4b1c-a232-500167ab2537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:24.535057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf87e28-a1de-4b1c-a232-500167ab2537 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:24.568840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 831/1665] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:24.569482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b28987-7161-4f4b-bb3b-14b18edc0d29 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:24.586722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b28987-7161-4f4b-bb3b-14b18edc0d29 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:24.615001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6ffec-d526-4aa9-adab-6d51a7351b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:24.616092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6ffec-d526-4aa9-adab-6d51a7351b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:24.616092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6ffec-d526-4aa9-adab-6d51a7351b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:24.616092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6ffec-d526-4aa9-adab-6d51a7351b2d admin admin] PATH_INFO: `/v3/projects/f1666fb7296945c4b3123dfc4bbfd568/users/42264ff0039b4135958d6b00728552ec/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:24.709220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6ffec-d526-4aa9-adab-6d51a7351b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1666fb7296945c4b3123dfc4bbfd568, user_id=42264ff0039b4135958d6b00728552ec, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:24.733134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6ffec-d526-4aa9-adab-6d51a7351b2d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:24.766722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 835/1666] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:24 2022] PUT /identity/v3/projects/f1666fb7296945c4b3123dfc4bbfd568/users/42264ff0039b4135958d6b00728552ec/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:24.770751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf87e28-a1de-4b1c-a232-500167ab2537 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:24.785715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf87e28-a1de-4b1c-a232-500167ab2537 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:24.808624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:24.998255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b90cc7-960d-4a8b-a82a-0a10213318a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:24.999041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b90cc7-960d-4a8b-a82a-0a10213318a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:24.999419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b90cc7-960d-4a8b-a82a-0a10213318a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:24.999802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b90cc7-960d-4a8b-a82a-0a10213318a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:25.000673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b90cc7-960d-4a8b-a82a-0a10213318a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:25.002544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b90cc7-960d-4a8b-a82a-0a10213318a6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:25.033192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 832/1667] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:25.044777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6ffec-d526-4aa9-adab-6d51a7351b2d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:25.065605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6ffec-d526-4aa9-adab-6d51a7351b2d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:25.082604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c6fb2-49e1-4034-a7e8-70d1d3cc4659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:25.087617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c6fb2-49e1-4034-a7e8-70d1d3cc4659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:25.088293 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c6fb2-49e1-4034-a7e8-70d1d3cc4659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:25.088293 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c6fb2-49e1-4034-a7e8-70d1d3cc4659 admin admin] PATH_INFO: `/v3/projects/f1666fb7296945c4b3123dfc4bbfd568/users/42264ff0039b4135958d6b00728552ec/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:25.109529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c6fb2-49e1-4034-a7e8-70d1d3cc4659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1666fb7296945c4b3123dfc4bbfd568, user_id=42264ff0039b4135958d6b00728552ec, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:25.116588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c6fb2-49e1-4034-a7e8-70d1d3cc4659 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:25.120048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b90cc7-960d-4a8b-a82a-0a10213318a6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:25.122279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-07b90cc7-960d-4a8b-a82a-0a10213318a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:25.137880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b90cc7-960d-4a8b-a82a-0a10213318a6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:25.140759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 836/1668] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:25 2022] PUT /identity/v3/projects/f1666fb7296945c4b3123dfc4bbfd568/users/42264ff0039b4135958d6b00728552ec/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:25.156759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8664566-59e9-4573-83ad-49e746d66b6f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:25.157541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8664566-59e9-4573-83ad-49e746d66b6f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:25.158039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8664566-59e9-4573-83ad-49e746d66b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:25.226151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:25.227475 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:25.230395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:25.231095 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] PATH_INFO: `/v3/users/baeb650373e84766a47f0825032a6648` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:25.236983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] RBAC: Authorizing `identity:delete_user(user_id=baeb650373e84766a47f0825032a6648)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:25.238353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:30:25.238353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 21 06:30:25.238353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: Jul 21 06:30:25.244257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:25.252763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b8664566-59e9-4573-83ad-49e746d66b6f None None] MFA Rules not processed for user `42264ff0039b4135958d6b00728552ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:25.257813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b8664566-59e9-4573-83ad-49e746d66b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:25.406651 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baeb650373e84766a47f0825032a6648'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:25.407368 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baeb650373e84766a47f0825032a6648'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:25.416969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:25.442596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baeb650373e84766a47f0825032a6648'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:25.504803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baeb650373e84766a47f0825032a6648'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:25.505505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cbe5e281-cb03-45e9-b795-ba7b285f8edc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baeb650373e84766a47f0825032a6648'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:25.517345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 837/1669] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:25.518751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3a73c-67f5-4a5e-99fb-988ca5bbe772 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:25.519046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3a73c-67f5-4a5e-99fb-988ca5bbe772 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:25.519328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3a73c-67f5-4a5e-99fb-988ca5bbe772 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:25.531884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 838/1670] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jul 21 06:30:25 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:30:25.533176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3a73c-67f5-4a5e-99fb-988ca5bbe772 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:25.539106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-23c3a73c-67f5-4a5e-99fb-988ca5bbe772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:25.550955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 833/1671] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:30:25 2022] DELETE /identity/v3/users/baeb650373e84766a47f0825032a6648 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:25.550955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee4aa7-5261-4cbd-8df4-29d778f3e871 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:25.553234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee4aa7-5261-4cbd-8df4-29d778f3e871 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:25.553801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee4aa7-5261-4cbd-8df4-29d778f3e871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:25.676966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-40ee4aa7-5261-4cbd-8df4-29d778f3e871 None None] MFA Rules not processed for user `1a9c3f34eaeb4b538528ff9050252dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:25.679401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-40ee4aa7-5261-4cbd-8df4-29d778f3e871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:25.682849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3a73c-67f5-4a5e-99fb-988ca5bbe772 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:25.697327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:25.815898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82830cc1-8d6c-49af-8c42-1a1a52526e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:25.816935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82830cc1-8d6c-49af-8c42-1a1a52526e31 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:25.818916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82830cc1-8d6c-49af-8c42-1a1a52526e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:25.819417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82830cc1-8d6c-49af-8c42-1a1a52526e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:25.819691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 834/1672] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:30:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:25.821171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b36325-9f9c-40fe-b9da-ddaf7dbc3537 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:25.821470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b36325-9f9c-40fe-b9da-ddaf7dbc3537 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:25.821751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b36325-9f9c-40fe-b9da-ddaf7dbc3537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:25.829663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-82830cc1-8d6c-49af-8c42-1a1a52526e31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:25.858241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d9b36325-9f9c-40fe-b9da-ddaf7dbc3537 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:25.859657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d9b36325-9f9c-40fe-b9da-ddaf7dbc3537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:25.966811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82830cc1-8d6c-49af-8c42-1a1a52526e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:25.968690 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82830cc1-8d6c-49af-8c42-1a1a52526e31 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:25.992915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 839/1673] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:25.996097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4eaee-d45e-47c7-a004-58f3790c214b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:25.996507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4eaee-d45e-47c7-a004-58f3790c214b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:25.999482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4eaee-d45e-47c7-a004-58f3790c214b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:26.036915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 835/1674] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:26.038007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b36325-9f9c-40fe-b9da-ddaf7dbc3537 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:26.049104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d9b36325-9f9c-40fe-b9da-ddaf7dbc3537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:26.078463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-afa4eaee-d45e-47c7-a004-58f3790c214b None None] MFA Rules not processed for user `8561734cf36e47b5ab2d7242076437ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:26.088563 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-afa4eaee-d45e-47c7-a004-58f3790c214b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:26.124159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b36325-9f9c-40fe-b9da-ddaf7dbc3537 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:26.146050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240e102-a788-44a5-a2fe-93990a3234cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:26.150223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240e102-a788-44a5-a2fe-93990a3234cf service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:26.150535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240e102-a788-44a5-a2fe-93990a3234cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:26.150906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240e102-a788-44a5-a2fe-93990a3234cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:26.189021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240e102-a788-44a5-a2fe-93990a3234cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:26.190525 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240e102-a788-44a5-a2fe-93990a3234cf service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:26.230337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 836/1675] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:26.230997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240e102-a788-44a5-a2fe-93990a3234cf service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:26.235432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c240e102-a788-44a5-a2fe-93990a3234cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:26.242826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:26.271339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240e102-a788-44a5-a2fe-93990a3234cf service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:26.287454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54eac6-4d4d-4989-9ae0-289ad1597308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:26.288135 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54eac6-4d4d-4989-9ae0-289ad1597308 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:26.288479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54eac6-4d4d-4989-9ae0-289ad1597308 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:26.288764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54eac6-4d4d-4989-9ae0-289ad1597308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:26.323652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54eac6-4d4d-4989-9ae0-289ad1597308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:26.325225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54eac6-4d4d-4989-9ae0-289ad1597308 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:26.360330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 837/1676] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:26.376371 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 840/1677] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:26.395182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54eac6-4d4d-4989-9ae0-289ad1597308 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:26.397829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8a54eac6-4d4d-4989-9ae0-289ad1597308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:26.636381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54eac6-4d4d-4989-9ae0-289ad1597308 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:26.661536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b91c4-869f-4ce5-bd11-01fcb815834c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:26.664238 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b91c4-869f-4ce5-bd11-01fcb815834c service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:26.664531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b91c4-869f-4ce5-bd11-01fcb815834c service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:26.669628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b91c4-869f-4ce5-bd11-01fcb815834c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:26.709421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b91c4-869f-4ce5-bd11-01fcb815834c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:26.715542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b91c4-869f-4ce5-bd11-01fcb815834c service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:26.767802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 838/1678] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:27.517666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbbe4a-05be-4cfa-b494-0e7c52b9008a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:27.518191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbbe4a-05be-4cfa-b494-0e7c52b9008a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:27.518372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbbe4a-05be-4cfa-b494-0e7c52b9008a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:27.519699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 841/1679] 10.222.0.42 () {54 vars in 961 bytes} [Thu Jul 21 06:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:30:27.534073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b958c-5127-47b7-8ecb-556e09d1879e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:27.534638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b958c-5127-47b7-8ecb-556e09d1879e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:27.534983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b958c-5127-47b7-8ecb-556e09d1879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:27.576231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-753b958c-5127-47b7-8ecb-556e09d1879e None None] MFA Rules not processed for user `1da2d85eb18049f196aeefc51d3f1e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:27.579132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-753b958c-5127-47b7-8ecb-556e09d1879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:27.613402 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:27.738763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64f5ae-47c6-4d7d-b9bb-ef8ebed79a56 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:27.751795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64f5ae-47c6-4d7d-b9bb-ef8ebed79a56 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:27.756651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64f5ae-47c6-4d7d-b9bb-ef8ebed79a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:27.835207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bf64f5ae-47c6-4d7d-b9bb-ef8ebed79a56 None None] MFA Rules not processed for user `774dd66c674a4955a439afd30354efab`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:27.836451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bf64f5ae-47c6-4d7d-b9bb-ef8ebed79a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:27.878834 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 839/1680] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Jul 21 06:30:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:27.890976 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b958c-5127-47b7-8ecb-556e09d1879e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:27.926296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b958c-5127-47b7-8ecb-556e09d1879e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:27.956757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a7a24-b246-4d34-8b60-a66aadde5011 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da2d85eb18049f196aeefc51d3f1e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:27.958511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a7a24-b246-4d34-8b60-a66aadde5011 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:27.958961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a7a24-b246-4d34-8b60-a66aadde5011 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:27.959447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a7a24-b246-4d34-8b60-a66aadde5011 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:27.963973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ad5a7a24-b246-4d34-8b60-a66aadde5011 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:27.989200 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a7a24-b246-4d34-8b60-a66aadde5011 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:27.993770 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a7a24-b246-4d34-8b60-a66aadde5011 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:28.034394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 840/1681] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:28.156346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 842/1682] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:29.785003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a7a24-b246-4d34-8b60-a66aadde5011 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:29.787296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ad5a7a24-b246-4d34-8b60-a66aadde5011 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:29.841003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a7a24-b246-4d34-8b60-a66aadde5011 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:29.873586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:29.875279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:29.875574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:29.880805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] PATH_INFO: `/v3/projects/c6346dc927ab4830b13d954cda6e048d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:29.886757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6346dc927ab4830b13d954cda6e048d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:29.899585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:30.160272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6346dc927ab4830b13d954cda6e048d', 'request_id': 'req-784ca7e8-9b0d-42ec-af0d-561ba6e58661'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:30.204146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6346dc927ab4830b13d954cda6e048d', 'request_id': 'req-784ca7e8-9b0d-42ec-af0d-561ba6e58661'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:30.226178 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6346dc927ab4830b13d954cda6e048d', 'request_id': 'req-784ca7e8-9b0d-42ec-af0d-561ba6e58661'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:30.277786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] The token cache is being invalidate because project c6346dc927ab4830b13d954cda6e048d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:30:30.278216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:30.285103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-784ca7e8-9b0d-42ec-af0d-561ba6e58661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:30.290745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 841/1683] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:30:29 2022] DELETE /identity/v3/projects/c6346dc927ab4830b13d954cda6e048d => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:30.296475 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca4fe3-84ef-4f7e-96a6-8e592e272b79 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:30.297006 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca4fe3-84ef-4f7e-96a6-8e592e272b79 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:30.297086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca4fe3-84ef-4f7e-96a6-8e592e272b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:30.359346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b2ca4fe3-84ef-4f7e-96a6-8e592e272b79 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:30.363041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b2ca4fe3-84ef-4f7e-96a6-8e592e272b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:30.554358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 843/1684] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:30.563228 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10281bd-ffb6-43d8-a1e9-896e78adf9d8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:30.563697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10281bd-ffb6-43d8-a1e9-896e78adf9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:30.564112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10281bd-ffb6-43d8-a1e9-896e78adf9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:30.611451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d10281bd-ffb6-43d8-a1e9-896e78adf9d8 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:30.612629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d10281bd-ffb6-43d8-a1e9-896e78adf9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:30.634473 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 842/1685] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:30.640645 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca4fe3-84ef-4f7e-96a6-8e592e272b79 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:30.661334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca4fe3-84ef-4f7e-96a6-8e592e272b79 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:30.685183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c2773-f447-461e-826c-c9da7cd5f116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:30.688284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c2773-f447-461e-826c-c9da7cd5f116 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:30.689369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c2773-f447-461e-826c-c9da7cd5f116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:30.690227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c2773-f447-461e-826c-c9da7cd5f116 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:30.691927 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c2773-f447-461e-826c-c9da7cd5f116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:30.693617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c2773-f447-461e-826c-c9da7cd5f116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:30.699289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c2773-f447-461e-826c-c9da7cd5f116 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:30.738358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 844/1686] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:30.739901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10281bd-ffb6-43d8-a1e9-896e78adf9d8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:30.755047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10281bd-ffb6-43d8-a1e9-896e78adf9d8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:30.769783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766c75d-7e6c-4040-98cc-d3e32d9e306e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:30.770035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766c75d-7e6c-4040-98cc-d3e32d9e306e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:30.775120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766c75d-7e6c-4040-98cc-d3e32d9e306e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:30.775927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766c75d-7e6c-4040-98cc-d3e32d9e306e admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:30.778887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766c75d-7e6c-4040-98cc-d3e32d9e306e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:30.783426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766c75d-7e6c-4040-98cc-d3e32d9e306e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:30.840664 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 843/1687] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:30 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:30.845242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c2773-f447-461e-826c-c9da7cd5f116 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:30.866328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c2773-f447-461e-826c-c9da7cd5f116 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:31.138783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0321c0-9a10-43d4-bc0f-f04d3abdbd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:31.140049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0321c0-9a10-43d4-bc0f-f04d3abdbd35 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:31.140049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0321c0-9a10-43d4-bc0f-f04d3abdbd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:31.140049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0321c0-9a10-43d4-bc0f-f04d3abdbd35 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:31.140695 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0321c0-9a10-43d4-bc0f-f04d3abdbd35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:31.149028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0321c0-9a10-43d4-bc0f-f04d3abdbd35 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:31.187926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:31.218132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 845/1688] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:30 2022] POST /identity/v3/users => generated 286 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:31.219713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766c75d-7e6c-4040-98cc-d3e32d9e306e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:31.238406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766c75d-7e6c-4040-98cc-d3e32d9e306e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:31.263821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0b112-f3c9-4701-baf4-fe3beb22dab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:31.264775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0b112-f3c9-4701-baf4-fe3beb22dab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:31.266294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0b112-f3c9-4701-baf4-fe3beb22dab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:31.268569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0b112-f3c9-4701-baf4-fe3beb22dab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:31.269543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0b112-f3c9-4701-baf4-fe3beb22dab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:31.274534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0b112-f3c9-4701-baf4-fe3beb22dab8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:31.307906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 844/1689] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:31.310669 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0321c0-9a10-43d4-bc0f-f04d3abdbd35 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:31.327911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0321c0-9a10-43d4-bc0f-f04d3abdbd35 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:31.336668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8612e-51a5-4836-bdbd-160e89a6e890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:31.340141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8612e-51a5-4836-bdbd-160e89a6e890 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:31.340716 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8612e-51a5-4836-bdbd-160e89a6e890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:31.341217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8612e-51a5-4836-bdbd-160e89a6e890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:31.343648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8612e-51a5-4836-bdbd-160e89a6e890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:31.345698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8612e-51a5-4836-bdbd-160e89a6e890 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:31.390077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0b112-f3c9-4701-baf4-fe3beb22dab8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:31.398046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 846/1690] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:31.406777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0b112-f3c9-4701-baf4-fe3beb22dab8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:31.427797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e77d31-1c83-46a9-9b6b-b4a0060c3f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:31.431359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e77d31-1c83-46a9-9b6b-b4a0060c3f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:31.431809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e77d31-1c83-46a9-9b6b-b4a0060c3f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:31.432175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e77d31-1c83-46a9-9b6b-b4a0060c3f5f admin admin] PATH_INFO: `/v3/projects/510a883c711246fd85309a230b68e97e/users/cc8c8b8b46dc4e4d91a415611f82ba23/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:31.450313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cd7ca-72e8-4111-b55a-18d316bc47c9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:31.450609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cd7ca-72e8-4111-b55a-18d316bc47c9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:31.450896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cd7ca-72e8-4111-b55a-18d316bc47c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:31.486086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e77d31-1c83-46a9-9b6b-b4a0060c3f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=510a883c711246fd85309a230b68e97e, user_id=cc8c8b8b46dc4e4d91a415611f82ba23, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:31.489803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e77d31-1c83-46a9-9b6b-b4a0060c3f5f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:31.522008 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 845/1691] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:31 2022] PUT /identity/v3/projects/510a883c711246fd85309a230b68e97e/users/cc8c8b8b46dc4e4d91a415611f82ba23/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:31.534396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bef70-86c4-4e89-8900-4aa91419b2cb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:31.534396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bef70-86c4-4e89-8900-4aa91419b2cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:31.535769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bef70-86c4-4e89-8900-4aa91419b2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:31.588430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c29cd7ca-72e8-4111-b55a-18d316bc47c9 None None] MFA Rules not processed for user `dfeb642f25484665b7c477cde153162d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:31.590235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c29cd7ca-72e8-4111-b55a-18d316bc47c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:31.643783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8d8bef70-86c4-4e89-8900-4aa91419b2cb None None] MFA Rules not processed for user `cc8c8b8b46dc4e4d91a415611f82ba23`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:31.654687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8d8bef70-86c4-4e89-8900-4aa91419b2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:31.690584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:31.914392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 847/1692] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:31.915772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cd7ca-72e8-4111-b55a-18d316bc47c9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:31.921046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c29cd7ca-72e8-4111-b55a-18d316bc47c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:31.982250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 846/1693] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:31.990848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0874b-1c66-4222-86b3-7a1d5d04d2dd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:31.991421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0874b-1c66-4222-86b3-7a1d5d04d2dd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:31.991711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0874b-1c66-4222-86b3-7a1d5d04d2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:32.090934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-79c0874b-1c66-4222-86b3-7a1d5d04d2dd None None] MFA Rules not processed for user `dfeb642f25484665b7c477cde153162d`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:32.095638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-79c0874b-1c66-4222-86b3-7a1d5d04d2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:32.140715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 847/1694] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:32.141828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0874b-1c66-4222-86b3-7a1d5d04d2dd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:32.169688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cd7ca-72e8-4111-b55a-18d316bc47c9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:32.186574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9f051-0981-4acb-869f-b78a6cc43377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:32.187486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9f051-0981-4acb-869f-b78a6cc43377 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:32.187839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9f051-0981-4acb-869f-b78a6cc43377 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:32.188069 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9f051-0981-4acb-869f-b78a6cc43377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:32.199417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-96c9f051-0981-4acb-869f-b78a6cc43377 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:32.535756 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9f051-0981-4acb-869f-b78a6cc43377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:32.537320 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9f051-0981-4acb-869f-b78a6cc43377 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:32.561995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 848/1695] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:32.565204 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9f051-0981-4acb-869f-b78a6cc43377 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:32.585287 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-96c9f051-0981-4acb-869f-b78a6cc43377 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:32.642741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0874b-1c66-4222-86b3-7a1d5d04d2dd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:32.698079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ac590-a3e8-440d-a728-0177e6fc3638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:32.698964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ac590-a3e8-440d-a728-0177e6fc3638 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:32.714640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ac590-a3e8-440d-a728-0177e6fc3638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:32.715013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ac590-a3e8-440d-a728-0177e6fc3638 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:32.715846 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ac590-a3e8-440d-a728-0177e6fc3638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:32.728548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ac590-a3e8-440d-a728-0177e6fc3638 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:32.772654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:32.871863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 848/1696] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:32 2022] POST /identity/v3/projects => generated 371 bytes in 730 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:32.877854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ac590-a3e8-440d-a728-0177e6fc3638 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:33.012152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9f051-0981-4acb-869f-b78a6cc43377 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:33.290376 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d358c-9706-4a20-8cb7-94a73a74af4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:33.291359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d358c-9706-4a20-8cb7-94a73a74af4e service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:33.291835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d358c-9706-4a20-8cb7-94a73a74af4e service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:33.292575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d358c-9706-4a20-8cb7-94a73a74af4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:33.322708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ac590-a3e8-440d-a728-0177e6fc3638 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:33.407061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab965a-acd0-46d8-851b-6a0bc39746c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:33.407790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab965a-acd0-46d8-851b-6a0bc39746c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:33.408075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab965a-acd0-46d8-851b-6a0bc39746c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:33.408356 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab965a-acd0-46d8-851b-6a0bc39746c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:33.409072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab965a-acd0-46d8-851b-6a0bc39746c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:33.425110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab965a-acd0-46d8-851b-6a0bc39746c5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:33.547853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 849/1697] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:32 2022] POST /identity/v3/users => generated 292 bytes in 671 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:33.549460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a8487-1175-4636-9c36-e3a78cf16686 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:33.549754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a8487-1175-4636-9c36-e3a78cf16686 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:33.558686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a8487-1175-4636-9c36-e3a78cf16686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:33.646068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d358c-9706-4a20-8cb7-94a73a74af4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:33.648495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d358c-9706-4a20-8cb7-94a73a74af4e service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:33.679223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9e4a8487-1175-4636-9c36-e3a78cf16686 None None] MFA Rules not processed for user `5c54ac39faa84c7db5b9b63360216d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:33.683466 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 849/1698] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 1121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:33.687914 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9e4a8487-1175-4636-9c36-e3a78cf16686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:33.688072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d358c-9706-4a20-8cb7-94a73a74af4e service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:33.771065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d358c-9706-4a20-8cb7-94a73a74af4e service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:33.778900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:33.806163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579326d5-c10f-45de-8b29-eaf15822ab95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:33.807836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579326d5-c10f-45de-8b29-eaf15822ab95 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:33.808107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579326d5-c10f-45de-8b29-eaf15822ab95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:33.808895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579326d5-c10f-45de-8b29-eaf15822ab95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:33.809147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579326d5-c10f-45de-8b29-eaf15822ab95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:33.828947 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579326d5-c10f-45de-8b29-eaf15822ab95 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:33.919157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 850/1699] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:33.931605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579326d5-c10f-45de-8b29-eaf15822ab95 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:33.986924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579326d5-c10f-45de-8b29-eaf15822ab95 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:34.010969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d775152-65e7-476a-8908-62c7a9c2c89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:34.011688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d775152-65e7-476a-8908-62c7a9c2c89d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:34.011977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d775152-65e7-476a-8908-62c7a9c2c89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:34.012261 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d775152-65e7-476a-8908-62c7a9c2c89d admin admin] PATH_INFO: `/v3/projects/0593b967f37d4186ae62959fea0ba9b5/users/130aab7797274965be30463a72e267b3/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:34.110558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d775152-65e7-476a-8908-62c7a9c2c89d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0593b967f37d4186ae62959fea0ba9b5, user_id=130aab7797274965be30463a72e267b3, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:34.117288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d775152-65e7-476a-8908-62c7a9c2c89d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:34.171661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 850/1700] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 623 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:34.179588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a8487-1175-4636-9c36-e3a78cf16686 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:34.188797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9e4a8487-1175-4636-9c36-e3a78cf16686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:34.195452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 851/1701] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:33 2022] PUT /identity/v3/projects/0593b967f37d4186ae62959fea0ba9b5/users/130aab7797274965be30463a72e267b3/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:34.200967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d775152-65e7-476a-8908-62c7a9c2c89d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:34.293177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:34.314729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a8487-1175-4636-9c36-e3a78cf16686 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:34.437881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d775152-65e7-476a-8908-62c7a9c2c89d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:34.473207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111c746-4fd9-46b4-86f9-175a98b4f8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:34.473790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111c746-4fd9-46b4-86f9-175a98b4f8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:34.475055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111c746-4fd9-46b4-86f9-175a98b4f8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:34.475055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111c746-4fd9-46b4-86f9-175a98b4f8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:34.475385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111c746-4fd9-46b4-86f9-175a98b4f8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:34.477537 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111c746-4fd9-46b4-86f9-175a98b4f8d1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:34.557039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 852/1702] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:34.565458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee7b69-43de-4c6a-9592-e11ef8d5f017 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:34.565458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee7b69-43de-4c6a-9592-e11ef8d5f017 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:34.565458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee7b69-43de-4c6a-9592-e11ef8d5f017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:34.710915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a792c9d-e212-44c2-8a18-49f4f0f9e416 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:34.714657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a792c9d-e212-44c2-8a18-49f4f0f9e416 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:34.717463 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a792c9d-e212-44c2-8a18-49f4f0f9e416 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:34.718279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a792c9d-e212-44c2-8a18-49f4f0f9e416 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:34.782251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-83ee7b69-43de-4c6a-9592-e11ef8d5f017 None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:34.783694 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-83ee7b69-43de-4c6a-9592-e11ef8d5f017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:35.042681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a792c9d-e212-44c2-8a18-49f4f0f9e416 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:35.049206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a792c9d-e212-44c2-8a18-49f4f0f9e416 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:35.082657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 853/1703] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:30:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 525 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:35.083791 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee7b69-43de-4c6a-9592-e11ef8d5f017 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:35.089572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 851/1704] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 911 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:35.092285 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a792c9d-e212-44c2-8a18-49f4f0f9e416 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:35.101541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8a792c9d-e212-44c2-8a18-49f4f0f9e416 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:35.127769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee7b69-43de-4c6a-9592-e11ef8d5f017 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:35.162606 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da871669-5ab9-4f75-8199-726c8d3a612b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:35.163667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871669-5ab9-4f75-8199-726c8d3a612b admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:35.167121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a792c9d-e212-44c2-8a18-49f4f0f9e416 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:35.171177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871669-5ab9-4f75-8199-726c8d3a612b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:35.171652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871669-5ab9-4f75-8199-726c8d3a612b admin admin] PATH_INFO: `/v3/projects/0593b967f37d4186ae62959fea0ba9b5/users/130aab7797274965be30463a72e267b3/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:35.187480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da871669-5ab9-4f75-8199-726c8d3a612b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0593b967f37d4186ae62959fea0ba9b5, user_id=130aab7797274965be30463a72e267b3, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:35.194895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da871669-5ab9-4f75-8199-726c8d3a612b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:35.205877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849822ee-aa78-4d3c-983b-51f4f25c4275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:35.208112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849822ee-aa78-4d3c-983b-51f4f25c4275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:35.213673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849822ee-aa78-4d3c-983b-51f4f25c4275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:35.214089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849822ee-aa78-4d3c-983b-51f4f25c4275 admin admin] PATH_INFO: `/v3/users/45e88807eb83454f8437224883b66c58` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:35.217297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849822ee-aa78-4d3c-983b-51f4f25c4275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e88807eb83454f8437224883b66c58)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:35.224548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849822ee-aa78-4d3c-983b-51f4f25c4275 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:35.286004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 854/1705] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:35 2022] PUT /identity/v3/projects/0593b967f37d4186ae62959fea0ba9b5/users/130aab7797274965be30463a72e267b3/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:35.292972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e13e0-e10c-4d83-b026-4e5e04ae8699 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:35.294403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e13e0-e10c-4d83-b026-4e5e04ae8699 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:35.295266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e13e0-e10c-4d83-b026-4e5e04ae8699 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:35.296792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 855/1706] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jul 21 06:30:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:30:35.303576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc091f-eea4-49f3-a2c4-97339b9cd121 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:35.305296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc091f-eea4-49f3-a2c4-97339b9cd121 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:35.305874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc091f-eea4-49f3-a2c4-97339b9cd121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:35.381888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:35.393142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-849822ee-aa78-4d3c-983b-51f4f25c4275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e88807eb83454f8437224883b66c58'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:35.394113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-849822ee-aa78-4d3c-983b-51f4f25c4275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e88807eb83454f8437224883b66c58'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:35.404607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-87bc091f-eea4-49f3-a2c4-97339b9cd121 None None] MFA Rules not processed for user `15ec5e26839f494e98ec70ac49ebe415`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:35.406992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-849822ee-aa78-4d3c-983b-51f4f25c4275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e88807eb83454f8437224883b66c58'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:35.408555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-87bc091f-eea4-49f3-a2c4-97339b9cd121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:35.424114 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-849822ee-aa78-4d3c-983b-51f4f25c4275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e88807eb83454f8437224883b66c58'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:35.426510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-849822ee-aa78-4d3c-983b-51f4f25c4275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e88807eb83454f8437224883b66c58'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:35.459150 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 852/1707] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:30:35 2022] DELETE /identity/v3/users/45e88807eb83454f8437224883b66c58 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:35.461219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633022f-a990-4ff8-ae9a-17b34877356b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:35.461790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633022f-a990-4ff8-ae9a-17b34877356b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:35.463399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633022f-a990-4ff8-ae9a-17b34877356b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:35.552572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5633022f-a990-4ff8-ae9a-17b34877356b None None] MFA Rules not processed for user `130aab7797274965be30463a72e267b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:35.553840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5633022f-a990-4ff8-ae9a-17b34877356b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:35.683210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 856/1708] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:35.686055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc58ce-dc35-4cbd-b8fd-d8027be30342 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:35.686471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc58ce-dc35-4cbd-b8fd-d8027be30342 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:35.686860 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc58ce-dc35-4cbd-b8fd-d8027be30342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:35.744449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-57fc58ce-dc35-4cbd-b8fd-d8027be30342 None None] MFA Rules not processed for user `1a9c3f34eaeb4b538528ff9050252dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:35.750081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-57fc58ce-dc35-4cbd-b8fd-d8027be30342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:35.771922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:35.915623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 857/1709] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:30:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:35.916947 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc58ce-dc35-4cbd-b8fd-d8027be30342 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:35.923387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-57fc58ce-dc35-4cbd-b8fd-d8027be30342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:35.930273 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 853/1710] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:35.942284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc972786-7eb8-4041-880d-2a73eae722ba None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:35.942624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc972786-7eb8-4041-880d-2a73eae722ba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:35.944474 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc972786-7eb8-4041-880d-2a73eae722ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:35.984559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dc972786-7eb8-4041-880d-2a73eae722ba None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:35.989196 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dc972786-7eb8-4041-880d-2a73eae722ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:36.158141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 854/1711] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:36.162271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc972786-7eb8-4041-880d-2a73eae722ba None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:36.163690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dc972786-7eb8-4041-880d-2a73eae722ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:36.229808 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:36.246353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc972786-7eb8-4041-880d-2a73eae722ba None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:36.247491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc58ce-dc35-4cbd-b8fd-d8027be30342 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:36.251795 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e643e7-2f78-4fc8-8d1f-a4848b1a552c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:36.252858 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e643e7-2f78-4fc8-8d1f-a4848b1a552c service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:36.253451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e643e7-2f78-4fc8-8d1f-a4848b1a552c service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:36.254968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e643e7-2f78-4fc8-8d1f-a4848b1a552c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:36.267929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e643e7-2f78-4fc8-8d1f-a4848b1a552c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:36.269528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e643e7-2f78-4fc8-8d1f-a4848b1a552c service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:36.276546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5578bcf-8440-4a31-9887-8d65bc20ee15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:36.277378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578bcf-8440-4a31-9887-8d65bc20ee15 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:36.278528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578bcf-8440-4a31-9887-8d65bc20ee15 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:36.278834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578bcf-8440-4a31-9887-8d65bc20ee15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:36.287259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d5578bcf-8440-4a31-9887-8d65bc20ee15 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:36.290232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 855/1712] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:36.292670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e643e7-2f78-4fc8-8d1f-a4848b1a552c service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:36.295145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-64e643e7-2f78-4fc8-8d1f-a4848b1a552c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:36.517066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5578bcf-8440-4a31-9887-8d65bc20ee15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:36.518583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e643e7-2f78-4fc8-8d1f-a4848b1a552c service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:36.518788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5578bcf-8440-4a31-9887-8d65bc20ee15 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:36.527902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93c0c3-e689-4e4b-a9f1-4584c88b3021 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:36.528977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93c0c3-e689-4e4b-a9f1-4584c88b3021 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:36.529521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93c0c3-e689-4e4b-a9f1-4584c88b3021 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:36.533346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93c0c3-e689-4e4b-a9f1-4584c88b3021 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:36.557737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 858/1713] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:30:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:30:36.574856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5578bcf-8440-4a31-9887-8d65bc20ee15 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:36.589235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d5578bcf-8440-4a31-9887-8d65bc20ee15 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:36.646324 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5578bcf-8440-4a31-9887-8d65bc20ee15 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:36.669999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd66928-6874-422d-a5db-912535b4df1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:36.677034 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd66928-6874-422d-a5db-912535b4df1d service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:36.678794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd66928-6874-422d-a5db-912535b4df1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:36.680508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd66928-6874-422d-a5db-912535b4df1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:36.775804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd66928-6874-422d-a5db-912535b4df1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:36.777784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd66928-6874-422d-a5db-912535b4df1d service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:36.832958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 859/1714] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:36.833866 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd66928-6874-422d-a5db-912535b4df1d service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:36.862478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93c0c3-e689-4e4b-a9f1-4584c88b3021 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:36.864109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93c0c3-e689-4e4b-a9f1-4584c88b3021 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:36.889846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd66928-6874-422d-a5db-912535b4df1d service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:36.906712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4d92-fd1e-487f-899a-b8ea362f08ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:36.907386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4d92-fd1e-487f-899a-b8ea362f08ee service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:36.907788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4d92-fd1e-487f-899a-b8ea362f08ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:36.907961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4d92-fd1e-487f-899a-b8ea362f08ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:36.910561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 856/1715] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:36.911512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93c0c3-e689-4e4b-a9f1-4584c88b3021 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:36.915310 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-044b4d92-fd1e-487f-899a-b8ea362f08ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:36.918994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6a93c0c3-e689-4e4b-a9f1-4584c88b3021 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:36.966059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93c0c3-e689-4e4b-a9f1-4584c88b3021 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:36.980000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aa0fd-5159-4962-8d42-11266661f266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:36.981951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aa0fd-5159-4962-8d42-11266661f266 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:36.982343 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aa0fd-5159-4962-8d42-11266661f266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:36.983338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aa0fd-5159-4962-8d42-11266661f266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:37.006091 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aa0fd-5159-4962-8d42-11266661f266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:37.007583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aa0fd-5159-4962-8d42-11266661f266 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:37.009062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:37.049975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 857/1716] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:37.060855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b7fca-ab0a-45c6-a282-75015c21430c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:37.061138 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b7fca-ab0a-45c6-a282-75015c21430c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:37.061418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b7fca-ab0a-45c6-a282-75015c21430c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:37.120820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-581b7fca-ab0a-45c6-a282-75015c21430c None None] MFA Rules not processed for user `8561734cf36e47b5ab2d7242076437ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:37.124576 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-581b7fca-ab0a-45c6-a282-75015c21430c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:37.132419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b4d92-fd1e-487f-899a-b8ea362f08ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:37.138023 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b4d92-fd1e-487f-899a-b8ea362f08ee service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:37.189629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 860/1717] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:37.413626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 858/1718] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:40.046508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4d92-fd1e-487f-899a-b8ea362f08ee service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:40.046508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-044b4d92-fd1e-487f-899a-b8ea362f08ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:40.124812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4d92-fd1e-487f-899a-b8ea362f08ee service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:40.179869 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:40.187118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:40.187561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:40.187967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] PATH_INFO: `/v3/projects/e8bedc0c93ca4c198b118bc51db92aa0` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:40.203346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8bedc0c93ca4c198b118bc51db92aa0)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:40.204523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 06:30:40.204523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 21 06:30:40.204523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: Jul 21 06:30:40.219278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:40.459896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8bedc0c93ca4c198b118bc51db92aa0', 'request_id': 'req-380606fe-67c1-463c-9410-02da7e75b055'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:40.491188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8bedc0c93ca4c198b118bc51db92aa0', 'request_id': 'req-380606fe-67c1-463c-9410-02da7e75b055'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:40.587025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8bedc0c93ca4c198b118bc51db92aa0', 'request_id': 'req-380606fe-67c1-463c-9410-02da7e75b055'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:40.658394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] The token cache is being invalidate because project e8bedc0c93ca4c198b118bc51db92aa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:30:40.658694 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:40.665080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-380606fe-67c1-463c-9410-02da7e75b055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:40.675981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 861/1719] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:30:40 2022] DELETE /identity/v3/projects/e8bedc0c93ca4c198b118bc51db92aa0 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:40.677900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b7fca-ab0a-45c6-a282-75015c21430c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:40.698334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b7fca-ab0a-45c6-a282-75015c21430c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:40.794563 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:41.050191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:41.051709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:41.052003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:41.052285 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] PATH_INFO: `/v3/users/29d74757a46b4b4c8dad8a0fb6a83d38` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:41.071410 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] RBAC: Authorizing `identity:delete_user(user_id=29d74757a46b4b4c8dad8a0fb6a83d38)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:41.078487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:41.256831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29d74757a46b4b4c8dad8a0fb6a83d38'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:41.258369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29d74757a46b4b4c8dad8a0fb6a83d38'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:41.298463 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29d74757a46b4b4c8dad8a0fb6a83d38'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:41.360030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29d74757a46b4b4c8dad8a0fb6a83d38'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:41.361878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29d74757a46b4b4c8dad8a0fb6a83d38'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:41.416875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 859/1720] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:30:40 2022] DELETE /identity/v3/users/29d74757a46b4b4c8dad8a0fb6a83d38 => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:44.326191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3824d0-9f42-428d-9baf-ef4a627ab708 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:44.326715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3824d0-9f42-428d-9baf-ef4a627ab708 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:44.326831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3824d0-9f42-428d-9baf-ef4a627ab708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:44.354900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6d3824d0-9f42-428d-9baf-ef4a627ab708 None None] MFA Rules not processed for user `5c54ac39faa84c7db5b9b63360216d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:44.356476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6d3824d0-9f42-428d-9baf-ef4a627ab708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:44.523499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:44.525043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:44.526420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 862/1721] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:44.698073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba757f62-ba08-4ee0-af6c-0da99713232d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:44.705919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:44.707103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:44.707596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:44.709310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] PATH_INFO: `/v3/projects/5f3d8cf6e6734049b474c2e1b60dd232` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:44.726775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f3d8cf6e6734049b474c2e1b60dd232)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:44.740839 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:44.888404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:44.903081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f3d8cf6e6734049b474c2e1b60dd232', 'request_id': 'req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:44.917406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f3d8cf6e6734049b474c2e1b60dd232', 'request_id': 'req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:44.930264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f3d8cf6e6734049b474c2e1b60dd232', 'request_id': 'req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:44.971718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] The token cache is being invalidate because project 5f3d8cf6e6734049b474c2e1b60dd232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:30:44.972407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:44.974071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2d5fd32-0e8c-47f8-8004-97f1a5f513c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:44.982349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bed4ed-3eb1-4c74-a856-283080ac8c9d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:44.983010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bed4ed-3eb1-4c74-a856-283080ac8c9d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:44.983758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bed4ed-3eb1-4c74-a856-283080ac8c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:44.985848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 860/1722] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:30:44 2022] DELETE /identity/v3/projects/5f3d8cf6e6734049b474c2e1b60dd232 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:45.010901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a5bed4ed-3eb1-4c74-a856-283080ac8c9d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:45.012834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a5bed4ed-3eb1-4c74-a856-283080ac8c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:45.080153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:45.111700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4db8f-1170-4eff-895d-c601d4491a8d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:45.112055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4db8f-1170-4eff-895d-c601d4491a8d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:45.112347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4db8f-1170-4eff-895d-c601d4491a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:45.124375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 863/1723] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:45.139874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d1c4db8f-1170-4eff-895d-c601d4491a8d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:45.141789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d1c4db8f-1170-4eff-895d-c601d4491a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:45.163831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bed4ed-3eb1-4c74-a856-283080ac8c9d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:45.176124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 861/1724] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:45.176590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bed4ed-3eb1-4c74-a856-283080ac8c9d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:45.182835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c8f33-c4e7-47ac-b001-23948434f77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:45.183545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c8f33-c4e7-47ac-b001-23948434f77a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:45.183833 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c8f33-c4e7-47ac-b001-23948434f77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:45.184111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c8f33-c4e7-47ac-b001-23948434f77a admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:45.184898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c8f33-c4e7-47ac-b001-23948434f77a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:45.185174 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c8f33-c4e7-47ac-b001-23948434f77a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:45.187213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c8f33-c4e7-47ac-b001-23948434f77a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:45.219485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 864/1725] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:45.224183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4db8f-1170-4eff-895d-c601d4491a8d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:45.236065 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4db8f-1170-4eff-895d-c601d4491a8d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:45.261972 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8ca4d-e3c5-44e6-8fd1-5fdee08bc5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:45.261972 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8ca4d-e3c5-44e6-8fd1-5fdee08bc5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:45.261972 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8ca4d-e3c5-44e6-8fd1-5fdee08bc5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:45.261972 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8ca4d-e3c5-44e6-8fd1-5fdee08bc5c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:45.262723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8ca4d-e3c5-44e6-8fd1-5fdee08bc5c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:45.262723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8ca4d-e3c5-44e6-8fd1-5fdee08bc5c5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:45.320095 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 862/1726] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:45 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:45.320958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c8f33-c4e7-47ac-b001-23948434f77a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:45.336641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c8f33-c4e7-47ac-b001-23948434f77a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:45.516064 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0def9-8953-4dcc-9af3-2ffa1e5d4ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:45.516781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0def9-8953-4dcc-9af3-2ffa1e5d4ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:45.517069 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0def9-8953-4dcc-9af3-2ffa1e5d4ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:45.520152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0def9-8953-4dcc-9af3-2ffa1e5d4ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:45.520916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0def9-8953-4dcc-9af3-2ffa1e5d4ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:45.522900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0def9-8953-4dcc-9af3-2ffa1e5d4ae6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:45.578544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8ca4d-e3c5-44e6-8fd1-5fdee08bc5c5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:45.589485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 865/1727] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:45 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:45.598744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8ca4d-e3c5-44e6-8fd1-5fdee08bc5c5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:45.614544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16664cc-70c0-4127-a369-bf0cfde28051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:45.615244 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16664cc-70c0-4127-a369-bf0cfde28051 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:45.615540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16664cc-70c0-4127-a369-bf0cfde28051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:45.615795 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16664cc-70c0-4127-a369-bf0cfde28051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:45.616598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16664cc-70c0-4127-a369-bf0cfde28051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:45.619372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16664cc-70c0-4127-a369-bf0cfde28051 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:45.673558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0def9-8953-4dcc-9af3-2ffa1e5d4ae6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:45.684204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 863/1728] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:30:45.685470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0def9-8953-4dcc-9af3-2ffa1e5d4ae6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:45.700828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a434263-42c7-4274-a964-a445d655f96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:45.701527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a434263-42c7-4274-a964-a445d655f96b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:45.701806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a434263-42c7-4274-a964-a445d655f96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:45.702107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a434263-42c7-4274-a964-a445d655f96b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:45.705289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a434263-42c7-4274-a964-a445d655f96b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:45.715665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a434263-42c7-4274-a964-a445d655f96b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:45.737160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16664cc-70c0-4127-a369-bf0cfde28051 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:45.744559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 866/1729] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:45.752940 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16664cc-70c0-4127-a369-bf0cfde28051 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:45.768552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c949f4-8b12-472a-891c-c814f6b9ef62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:45.769472 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c949f4-8b12-472a-891c-c814f6b9ef62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:45.769752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c949f4-8b12-472a-891c-c814f6b9ef62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:45.770073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c949f4-8b12-472a-891c-c814f6b9ef62 admin admin] PATH_INFO: `/v3/projects/db635a35c5cb43b296856fed215fd489/users/2158e614d4cf43169bfb91798dfc4e68/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:45.802339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c949f4-8b12-472a-891c-c814f6b9ef62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db635a35c5cb43b296856fed215fd489, user_id=2158e614d4cf43169bfb91798dfc4e68, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:45.805191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c949f4-8b12-472a-891c-c814f6b9ef62 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:45.840267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 864/1730] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:45 2022] PUT /identity/v3/projects/db635a35c5cb43b296856fed215fd489/users/2158e614d4cf43169bfb91798dfc4e68/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:45.851302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11661298-ba50-4823-87ec-5cd510261a99 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:45.851761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11661298-ba50-4823-87ec-5cd510261a99 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:45.852189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11661298-ba50-4823-87ec-5cd510261a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:45.956905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-11661298-ba50-4823-87ec-5cd510261a99 None None] MFA Rules not processed for user `2158e614d4cf43169bfb91798dfc4e68`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:45.972655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-11661298-ba50-4823-87ec-5cd510261a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:46.183930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 867/1731] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:46.188500 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c949f4-8b12-472a-891c-c814f6b9ef62 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:46.285338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c949f4-8b12-472a-891c-c814f6b9ef62 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:46.291631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f6261-4f1d-4533-80d4-3c887f6bc9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:46.292480 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f6261-4f1d-4533-80d4-3c887f6bc9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:46.292766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f6261-4f1d-4533-80d4-3c887f6bc9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:46.293045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f6261-4f1d-4533-80d4-3c887f6bc9f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:46.293767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f6261-4f1d-4533-80d4-3c887f6bc9f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:46.295870 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f6261-4f1d-4533-80d4-3c887f6bc9f1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:46.331466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 865/1732] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:46 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:46.334458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11661298-ba50-4823-87ec-5cd510261a99 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:46.347641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11661298-ba50-4823-87ec-5cd510261a99 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:46.426940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:46.445116 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c3b52-05d0-4e3c-8821-a178046fd557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:46.446222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c3b52-05d0-4e3c-8821-a178046fd557 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:46.447244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c3b52-05d0-4e3c-8821-a178046fd557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:46.447631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c3b52-05d0-4e3c-8821-a178046fd557 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:46.448442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c3b52-05d0-4e3c-8821-a178046fd557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:46.450786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c3b52-05d0-4e3c-8821-a178046fd557 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:46.495194 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 868/1733] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:46 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:30:46.499113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f6261-4f1d-4533-80d4-3c887f6bc9f1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:46.522186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f6261-4f1d-4533-80d4-3c887f6bc9f1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:46.532388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b4b60-d25a-4f9f-a585-46b1a4d65775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:46.535544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b4b60-d25a-4f9f-a585-46b1a4d65775 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:46.536121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b4b60-d25a-4f9f-a585-46b1a4d65775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:46.537222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b4b60-d25a-4f9f-a585-46b1a4d65775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:46.538356 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b4b60-d25a-4f9f-a585-46b1a4d65775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:46.542424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b4b60-d25a-4f9f-a585-46b1a4d65775 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:46.595032 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 866/1734] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:46.597416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c3b52-05d0-4e3c-8821-a178046fd557 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:46.619696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c3b52-05d0-4e3c-8821-a178046fd557 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:46.630838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0dfab8-3bd1-4419-96bf-c01fcfd2b183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:46.632742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dfab8-3bd1-4419-96bf-c01fcfd2b183 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:46.634574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dfab8-3bd1-4419-96bf-c01fcfd2b183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:46.636085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dfab8-3bd1-4419-96bf-c01fcfd2b183 admin admin] PATH_INFO: `/v3/projects/635c790fba0c4146a1c832ddbe1713af/users/c10bb4927cd549c8821dac1441cc09dd/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:46.672054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0dfab8-3bd1-4419-96bf-c01fcfd2b183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=635c790fba0c4146a1c832ddbe1713af, user_id=c10bb4927cd549c8821dac1441cc09dd, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:46.675169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0dfab8-3bd1-4419-96bf-c01fcfd2b183 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:46.697442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 869/1735] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:46 2022] PUT /identity/v3/projects/635c790fba0c4146a1c832ddbe1713af/users/c10bb4927cd549c8821dac1441cc09dd/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:46.699310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b4b60-d25a-4f9f-a585-46b1a4d65775 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:46.732726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b4b60-d25a-4f9f-a585-46b1a4d65775 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:46.802910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:46.840989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138090fd-7459-4e69-8f3b-6d503b2a4733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:46.841695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138090fd-7459-4e69-8f3b-6d503b2a4733 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:46.842023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138090fd-7459-4e69-8f3b-6d503b2a4733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:46.842572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138090fd-7459-4e69-8f3b-6d503b2a4733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:46.844135 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138090fd-7459-4e69-8f3b-6d503b2a4733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:46.845802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138090fd-7459-4e69-8f3b-6d503b2a4733 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:46.866614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 867/1736] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:46.867129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0dfab8-3bd1-4419-96bf-c01fcfd2b183 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:46.883079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0dfab8-3bd1-4419-96bf-c01fcfd2b183 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:46.916575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71b0d1-5a98-4a03-93d6-037ab605b346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:46.917517 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71b0d1-5a98-4a03-93d6-037ab605b346 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:46.919969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71b0d1-5a98-4a03-93d6-037ab605b346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:46.920278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71b0d1-5a98-4a03-93d6-037ab605b346 admin admin] PATH_INFO: `/v3/projects/635c790fba0c4146a1c832ddbe1713af/users/c10bb4927cd549c8821dac1441cc09dd/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:46.925506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71b0d1-5a98-4a03-93d6-037ab605b346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=635c790fba0c4146a1c832ddbe1713af, user_id=c10bb4927cd549c8821dac1441cc09dd, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:46.928444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71b0d1-5a98-4a03-93d6-037ab605b346 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:46.957054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 870/1737] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:46 2022] PUT /identity/v3/projects/635c790fba0c4146a1c832ddbe1713af/users/c10bb4927cd549c8821dac1441cc09dd/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:30:46.967626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fe603-3bec-4ce7-98bd-b466e30574f1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:46.967922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fe603-3bec-4ce7-98bd-b466e30574f1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:46.968550 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fe603-3bec-4ce7-98bd-b466e30574f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:47.022480 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e53fe603-3bec-4ce7-98bd-b466e30574f1 None None] MFA Rules not processed for user `c10bb4927cd549c8821dac1441cc09dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:47.037617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e53fe603-3bec-4ce7-98bd-b466e30574f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:47.255377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 868/1738] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:47.269198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71b0d1-5a98-4a03-93d6-037ab605b346 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:47.271515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fe71b0d1-5a98-4a03-93d6-037ab605b346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:47.349537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04e82e-62a6-4606-8943-ccfaab9bc48f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:47.350045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04e82e-62a6-4606-8943-ccfaab9bc48f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:47.350422 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04e82e-62a6-4606-8943-ccfaab9bc48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:47.383709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9d04e82e-62a6-4606-8943-ccfaab9bc48f None None] MFA Rules not processed for user `15ec5e26839f494e98ec70ac49ebe415`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:47.385353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9d04e82e-62a6-4606-8943-ccfaab9bc48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:47.421768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71b0d1-5a98-4a03-93d6-037ab605b346 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:47.433901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acb1ea-b9f5-407f-83d9-246283590bf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:47.434861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acb1ea-b9f5-407f-83d9-246283590bf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:47.435312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acb1ea-b9f5-407f-83d9-246283590bf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:47.435749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acb1ea-b9f5-407f-83d9-246283590bf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:47.449450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acb1ea-b9f5-407f-83d9-246283590bf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:47.452157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acb1ea-b9f5-407f-83d9-246283590bf6 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:47.495342 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 871/1739] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:47.535097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acb1ea-b9f5-407f-83d9-246283590bf6 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:47.536433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 869/1740] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:47.623732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acb1ea-b9f5-407f-83d9-246283590bf6 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:47.628699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:47.629599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:47.629883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:47.630192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] PATH_INFO: `/v3/users/2158e614d4cf43169bfb91798dfc4e68` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:47.631414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] RBAC: Authorizing `identity:delete_user(user_id=2158e614d4cf43169bfb91798dfc4e68)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:47.634123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:47.710961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:47.720296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2158e614d4cf43169bfb91798dfc4e68'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:47.744405 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2158e614d4cf43169bfb91798dfc4e68'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:47.753022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2158e614d4cf43169bfb91798dfc4e68'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:47.754537 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2158e614d4cf43169bfb91798dfc4e68'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:47.755588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2158e614d4cf43169bfb91798dfc4e68'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:47.780042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 872/1741] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:30:47 2022] DELETE /identity/v3/users/2158e614d4cf43169bfb91798dfc4e68 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:47.784711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c14360-fb1e-46ce-839e-6ce9822d29cc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:47.785214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c14360-fb1e-46ce-839e-6ce9822d29cc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:47.785318 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c14360-fb1e-46ce-839e-6ce9822d29cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:47.804555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e2c14360-fb1e-46ce-839e-6ce9822d29cc None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:47.806444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e2c14360-fb1e-46ce-839e-6ce9822d29cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:47.912049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:47.914802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:47.926919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 870/1742] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:30:48.031991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af1059-604a-42a2-b188-54a6333249df admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:48.035828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973e1e0-c0e5-42e0-8673-a9e3575c5015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:48.036554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e1e0-c0e5-42e0-8673-a9e3575c5015 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:48.037885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e1e0-c0e5-42e0-8673-a9e3575c5015 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:48.037885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e1e0-c0e5-42e0-8673-a9e3575c5015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:48.051819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973e1e0-c0e5-42e0-8673-a9e3575c5015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:48.054234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973e1e0-c0e5-42e0-8673-a9e3575c5015 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:48.091589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 873/1743] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:48.838115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c14360-fb1e-46ce-839e-6ce9822d29cc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:48.840171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e2c14360-fb1e-46ce-839e-6ce9822d29cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:48.849496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:48.872781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c14360-fb1e-46ce-839e-6ce9822d29cc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:48.889649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:48.890788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:48.892386 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:48.892885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] PATH_INFO: `/v3/projects/db635a35c5cb43b296856fed215fd489` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:48.900523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db635a35c5cb43b296856fed215fd489)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:48.903932 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:48.987346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d31d0-4ba8-448a-88a5-2af2cb4d004f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:48.991900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d31d0-4ba8-448a-88a5-2af2cb4d004f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:48.992630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d31d0-4ba8-448a-88a5-2af2cb4d004f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:49.064744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1a6d31d0-4ba8-448a-88a5-2af2cb4d004f None None] MFA Rules not processed for user `1a9c3f34eaeb4b538528ff9050252dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:49.082370 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1a6d31d0-4ba8-448a-88a5-2af2cb4d004f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:49.136713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db635a35c5cb43b296856fed215fd489', 'request_id': 'req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:49.168416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db635a35c5cb43b296856fed215fd489', 'request_id': 'req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:49.213237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db635a35c5cb43b296856fed215fd489', 'request_id': 'req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:49.258890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] The token cache is being invalidate because project db635a35c5cb43b296856fed215fd489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:30:49.259551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:49.265869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:49.276670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 871/1744] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:30:48 2022] DELETE /identity/v3/projects/db635a35c5cb43b296856fed215fd489 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:49.279593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:49.306042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:49.333566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 874/1745] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Jul 21 06:30:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:49.599515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4ec1b-3b8d-4bd8-9362-187f25611a5e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:49.629282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6cf71-91fa-4a86-9078-b22cef909764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:49.630490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6cf71-91fa-4a86-9078-b22cef909764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:49.632037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6cf71-91fa-4a86-9078-b22cef909764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:49.634864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6cf71-91fa-4a86-9078-b22cef909764 admin admin] PATH_INFO: `/v3/users/c10bb4927cd549c8821dac1441cc09dd` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:49.642084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6cf71-91fa-4a86-9078-b22cef909764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c10bb4927cd549c8821dac1441cc09dd)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:49.644169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6cf71-91fa-4a86-9078-b22cef909764 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:49.765471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-47a6cf71-91fa-4a86-9078-b22cef909764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c10bb4927cd549c8821dac1441cc09dd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:49.766472 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-47a6cf71-91fa-4a86-9078-b22cef909764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c10bb4927cd549c8821dac1441cc09dd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:49.787997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-47a6cf71-91fa-4a86-9078-b22cef909764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c10bb4927cd549c8821dac1441cc09dd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:49.813973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-47a6cf71-91fa-4a86-9078-b22cef909764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c10bb4927cd549c8821dac1441cc09dd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:49.815843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-47a6cf71-91fa-4a86-9078-b22cef909764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c10bb4927cd549c8821dac1441cc09dd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:49.843915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 872/1746] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:30:49 2022] DELETE /identity/v3/users/c10bb4927cd549c8821dac1441cc09dd => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:50.430628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d31d0-4ba8-448a-88a5-2af2cb4d004f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:50.453617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d31d0-4ba8-448a-88a5-2af2cb4d004f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:50.520037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:50.520948 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:50.521237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:50.521520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] PATH_INFO: `/v3/projects/635c790fba0c4146a1c832ddbe1713af` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:50.522800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=635c790fba0c4146a1c832ddbe1713af)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:50.524942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:50.544364 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:50.578839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '635c790fba0c4146a1c832ddbe1713af', 'request_id': 'req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:50.588446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '635c790fba0c4146a1c832ddbe1713af', 'request_id': 'req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:50.595456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '635c790fba0c4146a1c832ddbe1713af', 'request_id': 'req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:50.611256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] The token cache is being invalidate because project 635c790fba0c4146a1c832ddbe1713af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:30:50.611560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:50.612417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-44c2aa84-78d4-4dd9-878e-4f03bb2f5472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:30:50.618451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 875/1747] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:30:50 2022] DELETE /identity/v3/projects/635c790fba0c4146a1c832ddbe1713af => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:50.620146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203976d-e705-4972-99d0-74bbdbdbd45c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:50.620444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203976d-e705-4972-99d0-74bbdbdbd45c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:50.620723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203976d-e705-4972-99d0-74bbdbdbd45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:50.631935 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:50.651880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2203976d-e705-4972-99d0-74bbdbdbd45c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:50.653632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2203976d-e705-4972-99d0-74bbdbdbd45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:50.744005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 873/1748] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:50.753369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d820b04-2491-4f4e-976d-5728f6bb03c1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:50.753905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d820b04-2491-4f4e-976d-5728f6bb03c1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:50.754369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d820b04-2491-4f4e-976d-5728f6bb03c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:50.775088 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0d820b04-2491-4f4e-976d-5728f6bb03c1 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:50.776328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0d820b04-2491-4f4e-976d-5728f6bb03c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:50.792283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 876/1749] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:50.803759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203976d-e705-4972-99d0-74bbdbdbd45c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:50.831366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203976d-e705-4972-99d0-74bbdbdbd45c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:50.837039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c9c3c-b0ec-4f26-b340-5c44fcc80e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:50.838029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c9c3c-b0ec-4f26-b340-5c44fcc80e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:50.838277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c9c3c-b0ec-4f26-b340-5c44fcc80e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:50.838556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c9c3c-b0ec-4f26-b340-5c44fcc80e38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:50.839297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c9c3c-b0ec-4f26-b340-5c44fcc80e38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:30:50.839604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c9c3c-b0ec-4f26-b340-5c44fcc80e38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:50.841204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c9c3c-b0ec-4f26-b340-5c44fcc80e38 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:50.861918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 874/1750] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:30:50.862599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d820b04-2491-4f4e-976d-5728f6bb03c1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:50.878972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d820b04-2491-4f4e-976d-5728f6bb03c1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:50.885335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc15a2b-10ce-4e1c-819e-f4f6094e1b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:50.886527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc15a2b-10ce-4e1c-819e-f4f6094e1b62 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:50.886981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc15a2b-10ce-4e1c-819e-f4f6094e1b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:50.887456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc15a2b-10ce-4e1c-819e-f4f6094e1b62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:50.888277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc15a2b-10ce-4e1c-819e-f4f6094e1b62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:50.891477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc15a2b-10ce-4e1c-819e-f4f6094e1b62 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:50.960379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 877/1751] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:50 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:30:50.964772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c9c3c-b0ec-4f26-b340-5c44fcc80e38 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:50.984743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c9c3c-b0ec-4f26-b340-5c44fcc80e38 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:51.066592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4507c8-b694-462a-b3b6-3fd0fe2427a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:51.067574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4507c8-b694-462a-b3b6-3fd0fe2427a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.068057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4507c8-b694-462a-b3b6-3fd0fe2427a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.068521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4507c8-b694-462a-b3b6-3fd0fe2427a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.069446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4507c8-b694-462a-b3b6-3fd0fe2427a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:51.072923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4507c8-b694-462a-b3b6-3fd0fe2427a0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:51.119094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 875/1752] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:50 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:51.121142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc15a2b-10ce-4e1c-819e-f4f6094e1b62 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:51.132473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc15a2b-10ce-4e1c-819e-f4f6094e1b62 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:51.139134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c5172-be9b-485a-a17a-bb4513f850fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:51.139831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c5172-be9b-485a-a17a-bb4513f850fe admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.140105 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c5172-be9b-485a-a17a-bb4513f850fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.140886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c5172-be9b-485a-a17a-bb4513f850fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.141135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c5172-be9b-485a-a17a-bb4513f850fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:51.144800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c5172-be9b-485a-a17a-bb4513f850fe admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:51.161471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4507c8-b694-462a-b3b6-3fd0fe2427a0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:51.162453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 878/1753] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:51.177251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4507c8-b694-462a-b3b6-3fd0fe2427a0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:51.184099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c9f35-cca5-44cb-993b-1a1b8dc5d0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:51.184798 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c9f35-cca5-44cb-993b-1a1b8dc5d0de admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.185510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c9f35-cca5-44cb-993b-1a1b8dc5d0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.186091 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c9f35-cca5-44cb-993b-1a1b8dc5d0de admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.187519 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c9f35-cca5-44cb-993b-1a1b8dc5d0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:51.189711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c9f35-cca5-44cb-993b-1a1b8dc5d0de admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:51.206843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 876/1754] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:30:51.211068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c5172-be9b-485a-a17a-bb4513f850fe admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:51.223156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c5172-be9b-485a-a17a-bb4513f850fe admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:51.228325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042b6c0-ced5-4d7b-b40a-d14492916b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:51.229220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042b6c0-ced5-4d7b-b40a-d14492916b89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.229510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042b6c0-ced5-4d7b-b40a-d14492916b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.229790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042b6c0-ced5-4d7b-b40a-d14492916b89 admin admin] PATH_INFO: `/v3/projects/3c98c6ff51334e92b65da30bbae780a5/users/f06ee96981854149bfb8c06170f3149f/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.253011 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042b6c0-ced5-4d7b-b40a-d14492916b89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c98c6ff51334e92b65da30bbae780a5, user_id=f06ee96981854149bfb8c06170f3149f, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:51.256005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042b6c0-ced5-4d7b-b40a-d14492916b89 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:51.279684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 879/1755] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:51 2022] PUT /identity/v3/projects/3c98c6ff51334e92b65da30bbae780a5/users/f06ee96981854149bfb8c06170f3149f/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:51.290528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbbe90-2f3d-4320-913e-0e2265060633 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.290816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbbe90-2f3d-4320-913e-0e2265060633 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.291294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbbe90-2f3d-4320-913e-0e2265060633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.320997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-defbbe90-2f3d-4320-913e-0e2265060633 None None] MFA Rules not processed for user `f06ee96981854149bfb8c06170f3149f`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:51.323334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-defbbe90-2f3d-4320-913e-0e2265060633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:51.417266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:51.426516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 877/1756] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:51.427227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042b6c0-ced5-4d7b-b40a-d14492916b89 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:51.510090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042b6c0-ced5-4d7b-b40a-d14492916b89 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:51.516093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e788264-b3e8-4c9f-933c-862482f37850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:51.516881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e788264-b3e8-4c9f-933c-862482f37850 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.517264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e788264-b3e8-4c9f-933c-862482f37850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.517634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e788264-b3e8-4c9f-933c-862482f37850 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.518585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e788264-b3e8-4c9f-933c-862482f37850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:51.520917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e788264-b3e8-4c9f-933c-862482f37850 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:51.561113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 880/1757] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:51 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:51.565203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbbe90-2f3d-4320-913e-0e2265060633 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:51.581444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbbe90-2f3d-4320-913e-0e2265060633 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:51.640948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534eec2-7822-49b0-b9d7-65e3fb7ffff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:51.641833 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534eec2-7822-49b0-b9d7-65e3fb7ffff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.642153 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534eec2-7822-49b0-b9d7-65e3fb7ffff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.642434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534eec2-7822-49b0-b9d7-65e3fb7ffff7 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.643239 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534eec2-7822-49b0-b9d7-65e3fb7ffff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:51.645160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534eec2-7822-49b0-b9d7-65e3fb7ffff7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:51.667691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 878/1758] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:51 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:51.671443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e788264-b3e8-4c9f-933c-862482f37850 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:51.681863 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e788264-b3e8-4c9f-933c-862482f37850 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:51.686527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c17424-f029-411e-9429-5822eb542ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:51.687202 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c17424-f029-411e-9429-5822eb542ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.687488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c17424-f029-411e-9429-5822eb542ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.687766 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c17424-f029-411e-9429-5822eb542ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.688504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c17424-f029-411e-9429-5822eb542ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:51.690161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c17424-f029-411e-9429-5822eb542ec6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:51.702169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 881/1759] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:51.706141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534eec2-7822-49b0-b9d7-65e3fb7ffff7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:51.718175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534eec2-7822-49b0-b9d7-65e3fb7ffff7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:51.722785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba43c2-a81b-4918-aa4f-61d409dbda8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:51.723460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba43c2-a81b-4918-aa4f-61d409dbda8e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.723746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba43c2-a81b-4918-aa4f-61d409dbda8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.724029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba43c2-a81b-4918-aa4f-61d409dbda8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.724773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba43c2-a81b-4918-aa4f-61d409dbda8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:51.726453 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba43c2-a81b-4918-aa4f-61d409dbda8e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:51.737208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 879/1760] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:51.741449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c17424-f029-411e-9429-5822eb542ec6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:51.751614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c17424-f029-411e-9429-5822eb542ec6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:51.756122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b4201-f393-4dbd-8c21-fa2ebd9e85ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:51.757005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b4201-f393-4dbd-8c21-fa2ebd9e85ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.757292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b4201-f393-4dbd-8c21-fa2ebd9e85ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.757566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b4201-f393-4dbd-8c21-fa2ebd9e85ab admin admin] PATH_INFO: `/v3/projects/41c4a14a445a4fdc8f553e322cfdbb9b/users/6e2f759edf4a4b3491b8dbb31b078834/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.769569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:51.773044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b4201-f393-4dbd-8c21-fa2ebd9e85ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=41c4a14a445a4fdc8f553e322cfdbb9b, user_id=6e2f759edf4a4b3491b8dbb31b078834, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:51.776970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b4201-f393-4dbd-8c21-fa2ebd9e85ab admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:51.803787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 882/1761] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:51 2022] PUT /identity/v3/projects/41c4a14a445a4fdc8f553e322cfdbb9b/users/6e2f759edf4a4b3491b8dbb31b078834/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:51.806013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af4100-f15c-4e12-aed8-63b513d8ded0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.806411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af4100-f15c-4e12-aed8-63b513d8ded0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.806827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af4100-f15c-4e12-aed8-63b513d8ded0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.834822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f2af4100-f15c-4e12-aed8-63b513d8ded0 None None] MFA Rules not processed for user `6e2f759edf4a4b3491b8dbb31b078834`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:51.836145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f2af4100-f15c-4e12-aed8-63b513d8ded0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:51.907999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 880/1762] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:51.912661 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b4201-f393-4dbd-8c21-fa2ebd9e85ab admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:51.975967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b4201-f393-4dbd-8c21-fa2ebd9e85ab admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:51.982039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf5552-0d2c-4810-b87e-0d8206665c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:51.982877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf5552-0d2c-4810-b87e-0d8206665c68 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:51.983165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf5552-0d2c-4810-b87e-0d8206665c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:51.983441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf5552-0d2c-4810-b87e-0d8206665c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:51.985331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf5552-0d2c-4810-b87e-0d8206665c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:51.987260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf5552-0d2c-4810-b87e-0d8206665c68 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:52.015312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 883/1763] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:30:51 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:52.020096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af4100-f15c-4e12-aed8-63b513d8ded0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:52.034644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af4100-f15c-4e12-aed8-63b513d8ded0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:52.111506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaecec-c4ab-447c-9994-29621b2bec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:52.112203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaecec-c4ab-447c-9994-29621b2bec1d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:52.112486 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaecec-c4ab-447c-9994-29621b2bec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:52.112764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaecec-c4ab-447c-9994-29621b2bec1d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:52.113470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafaecec-c4ab-447c-9994-29621b2bec1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:52.115558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafaecec-c4ab-447c-9994-29621b2bec1d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:52.138912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:52.147153 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 881/1764] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:30:52 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:30:52.151768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf5552-0d2c-4810-b87e-0d8206665c68 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:52.163356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf5552-0d2c-4810-b87e-0d8206665c68 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:52.169422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca38db-a303-4748-9611-85ea9d5b8a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:52.170379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca38db-a303-4748-9611-85ea9d5b8a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:52.170729 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca38db-a303-4748-9611-85ea9d5b8a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:52.171079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca38db-a303-4748-9611-85ea9d5b8a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:52.171889 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca38db-a303-4748-9611-85ea9d5b8a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:52.173596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca38db-a303-4748-9611-85ea9d5b8a32 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:52.187874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 884/1765] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:52.191968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaecec-c4ab-447c-9994-29621b2bec1d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:52.203764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaecec-c4ab-447c-9994-29621b2bec1d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:52.210908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c074db-b324-4813-9a89-9bae8176be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:52.210908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c074db-b324-4813-9a89-9bae8176be3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:52.210908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c074db-b324-4813-9a89-9bae8176be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:52.210908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c074db-b324-4813-9a89-9bae8176be3b admin admin] PATH_INFO: `/v3/projects/e002fe23cfac48948fcb0de1f21e9dc8/users/d33aad3725c645388274d4400a4e9ef0/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:52.244186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c074db-b324-4813-9a89-9bae8176be3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e002fe23cfac48948fcb0de1f21e9dc8, user_id=d33aad3725c645388274d4400a4e9ef0, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:52.247097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c074db-b324-4813-9a89-9bae8176be3b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:52.270657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 882/1766] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:52 2022] PUT /identity/v3/projects/e002fe23cfac48948fcb0de1f21e9dc8/users/d33aad3725c645388274d4400a4e9ef0/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:52.275539 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca38db-a303-4748-9611-85ea9d5b8a32 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:52.287466 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca38db-a303-4748-9611-85ea9d5b8a32 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:52.406883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36f50d-e389-4fcd-9eb5-d7ae98c7c222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:52.407889 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36f50d-e389-4fcd-9eb5-d7ae98c7c222 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:52.408297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36f50d-e389-4fcd-9eb5-d7ae98c7c222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:52.408652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36f50d-e389-4fcd-9eb5-d7ae98c7c222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:52.409706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36f50d-e389-4fcd-9eb5-d7ae98c7c222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:52.416719 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36f50d-e389-4fcd-9eb5-d7ae98c7c222 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:52.438176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 885/1767] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:30:52.442499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c074db-b324-4813-9a89-9bae8176be3b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:52.457741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c074db-b324-4813-9a89-9bae8176be3b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:52.466044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a17d52-8bae-4897-bc1c-6b999befd4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:52.466949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a17d52-8bae-4897-bc1c-6b999befd4f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:52.467232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a17d52-8bae-4897-bc1c-6b999befd4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:52.467513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a17d52-8bae-4897-bc1c-6b999befd4f3 admin admin] PATH_INFO: `/v3/projects/e002fe23cfac48948fcb0de1f21e9dc8/users/d33aad3725c645388274d4400a4e9ef0/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:52.470428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a17d52-8bae-4897-bc1c-6b999befd4f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e002fe23cfac48948fcb0de1f21e9dc8, user_id=d33aad3725c645388274d4400a4e9ef0, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:52.473479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a17d52-8bae-4897-bc1c-6b999befd4f3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:52.486991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 883/1768] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:30:52 2022] PUT /identity/v3/projects/e002fe23cfac48948fcb0de1f21e9dc8/users/d33aad3725c645388274d4400a4e9ef0/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:30:52.498501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0c75d-1f00-4a16-b9c8-b12128ed66dd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:52.499077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0c75d-1f00-4a16-b9c8-b12128ed66dd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:52.499510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0c75d-1f00-4a16-b9c8-b12128ed66dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:52.539023 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-32d0c75d-1f00-4a16-b9c8-b12128ed66dd None None] MFA Rules not processed for user `d33aad3725c645388274d4400a4e9ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:30:52.542004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-32d0c75d-1f00-4a16-b9c8-b12128ed66dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:52.668851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:30:52.749042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 886/1769] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:30:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:30:52.760534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a17d52-8bae-4897-bc1c-6b999befd4f3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:52.764436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e1a17d52-8bae-4897-bc1c-6b999befd4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:52.883531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a17d52-8bae-4897-bc1c-6b999befd4f3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:52.901982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc2ec3-dd22-4777-8177-f06bbd4a38e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:52.901982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc2ec3-dd22-4777-8177-f06bbd4a38e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:52.901982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc2ec3-dd22-4777-8177-f06bbd4a38e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:52.901982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc2ec3-dd22-4777-8177-f06bbd4a38e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:52.917043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc2ec3-dd22-4777-8177-f06bbd4a38e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:52.918924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc2ec3-dd22-4777-8177-f06bbd4a38e6 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:52.929776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 884/1770] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:53.876393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0c75d-1f00-4a16-b9c8-b12128ed66dd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:53.879493 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-32d0c75d-1f00-4a16-b9c8-b12128ed66dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:53.897596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0c75d-1f00-4a16-b9c8-b12128ed66dd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:53.903952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c7314-144e-466b-b1d2-6ee2c87f438f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:53.904771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c7314-144e-466b-b1d2-6ee2c87f438f service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:53.905365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c7314-144e-466b-b1d2-6ee2c87f438f service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:53.905788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c7314-144e-466b-b1d2-6ee2c87f438f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:54.028720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c7314-144e-466b-b1d2-6ee2c87f438f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:54.030606 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c7314-144e-466b-b1d2-6ee2c87f438f service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:54.045304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 887/1771] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:58.138608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc2ec3-dd22-4777-8177-f06bbd4a38e6 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:58.139116 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c7314-144e-466b-b1d2-6ee2c87f438f service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:30:58.140362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-931c7314-144e-466b-b1d2-6ee2c87f438f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:58.140448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1bdc2ec3-dd22-4777-8177-f06bbd4a38e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:58.221669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc2ec3-dd22-4777-8177-f06bbd4a38e6 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:58.226102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cfe14-4c79-4ca9-82df-1bcca6b63a7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:58.227751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cfe14-4c79-4ca9-82df-1bcca6b63a7f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:58.228084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cfe14-4c79-4ca9-82df-1bcca6b63a7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:58.228376 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cfe14-4c79-4ca9-82df-1bcca6b63a7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:58.230512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6d4cfe14-4c79-4ca9-82df-1bcca6b63a7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:30:58.233970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c7314-144e-466b-b1d2-6ee2c87f438f service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:30:58.238485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3e88f-aa0c-4be4-a85b-361f8bdcb6db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:30:58.239714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3e88f-aa0c-4be4-a85b-361f8bdcb6db service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:30:58.240162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3e88f-aa0c-4be4-a85b-361f8bdcb6db service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:30:58.240589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3e88f-aa0c-4be4-a85b-361f8bdcb6db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:30:58.245334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4cfe14-4c79-4ca9-82df-1bcca6b63a7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:58.247908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4cfe14-4c79-4ca9-82df-1bcca6b63a7f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:58.251499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3e88f-aa0c-4be4-a85b-361f8bdcb6db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:30:58.253708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3e88f-aa0c-4be4-a85b-361f8bdcb6db service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:30:58.260294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 885/1772] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:30:58.284737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 888/1773] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:30:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:31:00.574526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cfe14-4c79-4ca9-82df-1bcca6b63a7f service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:00.579852 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47890427-08f5-43da-a4a7-93ca3d2a1a54 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:00.580351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47890427-08f5-43da-a4a7-93ca3d2a1a54 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:00.580457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47890427-08f5-43da-a4a7-93ca3d2a1a54 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:00.581810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 889/1774] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 06:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:31:00.592759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cfe14-4c79-4ca9-82df-1bcca6b63a7f service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:00.597207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e975d6-3361-4406-95f4-494ed5826fc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:00.599174 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e975d6-3361-4406-95f4-494ed5826fc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:00.600850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e975d6-3361-4406-95f4-494ed5826fc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:00.601845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e975d6-3361-4406-95f4-494ed5826fc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:00.709160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e975d6-3361-4406-95f4-494ed5826fc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:00.709993 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbc76d-09a3-4063-9130-541d82a1a9b4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:00.710555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbc76d-09a3-4063-9130-541d82a1a9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:00.711090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbc76d-09a3-4063-9130-541d82a1a9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:00.711830 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e975d6-3361-4406-95f4-494ed5826fc4 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:00.744944 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-59fbc76d-09a3-4063-9130-541d82a1a9b4 None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:00.746078 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-59fbc76d-09a3-4063-9130-541d82a1a9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:00.788731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 886/1775] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:00.903436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 890/1776] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 06:31:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:01.017470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e975d6-3361-4406-95f4-494ed5826fc4 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:01.021820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e3e975d6-3361-4406-95f4-494ed5826fc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:01.048200 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:01.076012 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e975d6-3361-4406-95f4-494ed5826fc4 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:01.096561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3577fa6-d4b8-4de5-9b79-0872863aef1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:01.097669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3577fa6-d4b8-4de5-9b79-0872863aef1f service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:01.098137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3577fa6-d4b8-4de5-9b79-0872863aef1f service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:01.098524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3577fa6-d4b8-4de5-9b79-0872863aef1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:01.142389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3577fa6-d4b8-4de5-9b79-0872863aef1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:01.144724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3577fa6-d4b8-4de5-9b79-0872863aef1f service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:01.173143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 887/1777] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:02.547243 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043ef03-5311-4e31-bc2b-84f6bf2ca46e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:02.547765 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043ef03-5311-4e31-bc2b-84f6bf2ca46e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:02.547878 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043ef03-5311-4e31-bc2b-84f6bf2ca46e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:02.549256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 891/1778] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 06:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:31:02.582873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f06c54-c63d-44a1-8d70-845d8ee26255 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:02.583321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f06c54-c63d-44a1-8d70-845d8ee26255 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:02.583442 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f06c54-c63d-44a1-8d70-845d8ee26255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:02.608771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-03f06c54-c63d-44a1-8d70-845d8ee26255 None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:02.610394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-03f06c54-c63d-44a1-8d70-845d8ee26255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:02.625074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 888/1779] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 06:31:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:02.927235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043ef03-5311-4e31-bc2b-84f6bf2ca46e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:02.931592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0043ef03-5311-4e31-bc2b-84f6bf2ca46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:02.951640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043ef03-5311-4e31-bc2b-84f6bf2ca46e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:02.959891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207c8e7-e047-47b8-9e74-e514a4f03971 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:02.961444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207c8e7-e047-47b8-9e74-e514a4f03971 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:02.962301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207c8e7-e047-47b8-9e74-e514a4f03971 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:02.962774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207c8e7-e047-47b8-9e74-e514a4f03971 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:02.976987 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207c8e7-e047-47b8-9e74-e514a4f03971 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:02.978718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207c8e7-e047-47b8-9e74-e514a4f03971 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:02.994472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 892/1780] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:04.111025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f06c54-c63d-44a1-8d70-845d8ee26255 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:04.248467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f06c54-c63d-44a1-8d70-845d8ee26255 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:04.255887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:04.257434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:04.258396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:04.258844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] PATH_INFO: `/v3/users/f06ee96981854149bfb8c06170f3149f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:04.261414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06ee96981854149bfb8c06170f3149f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:04.264246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:04.347577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06ee96981854149bfb8c06170f3149f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:04.348767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06ee96981854149bfb8c06170f3149f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:04.362243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06ee96981854149bfb8c06170f3149f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:04.381247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06ee96981854149bfb8c06170f3149f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:04.383332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06ee96981854149bfb8c06170f3149f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:04.409307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 889/1781] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:04 2022] DELETE /identity/v3/users/f06ee96981854149bfb8c06170f3149f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:04.415789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6e10f-8f48-4284-8e2c-7f29f576ddfd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:04.417696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6e10f-8f48-4284-8e2c-7f29f576ddfd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:04.418307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6e10f-8f48-4284-8e2c-7f29f576ddfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:04.440483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:04.451099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-08d6e10f-8f48-4284-8e2c-7f29f576ddfd None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:04.453036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-08d6e10f-8f48-4284-8e2c-7f29f576ddfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:04.562800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 893/1782] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:04.564646 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:04.566804 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:04.578245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6e10f-8f48-4284-8e2c-7f29f576ddfd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:04.586411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-08d6e10f-8f48-4284-8e2c-7f29f576ddfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:04.594113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18524601-938e-41d1-8f20-9a0cde25c484 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:04.606247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:04.621104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:04.621104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:04.621601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] PATH_INFO: `/v3/users/b8d4d9874077426297e6eb717d558f45` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:04.623135 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d4d9874077426297e6eb717d558f45)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:04.625372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:04.685551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6e10f-8f48-4284-8e2c-7f29f576ddfd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:04.690028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70958a5-2e87-4cb9-91b6-80a99538be1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:04.690699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70958a5-2e87-4cb9-91b6-80a99538be1d service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:04.690980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70958a5-2e87-4cb9-91b6-80a99538be1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:04.691256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70958a5-2e87-4cb9-91b6-80a99538be1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:04.713602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70958a5-2e87-4cb9-91b6-80a99538be1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:04.714063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d4d9874077426297e6eb717d558f45'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:04.715150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70958a5-2e87-4cb9-91b6-80a99538be1d service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:04.715629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d4d9874077426297e6eb717d558f45'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:04.732700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 894/1783] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:31:04.736540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d4d9874077426297e6eb717d558f45'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:04.740580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:04.759352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d4d9874077426297e6eb717d558f45'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:04.760113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d4d9874077426297e6eb717d558f45'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:04.780963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 890/1784] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:04 2022] DELETE /identity/v3/users/b8d4d9874077426297e6eb717d558f45 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:04.785627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78d77a-6e32-4a07-ac1f-85e98865d817 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:04.785763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78d77a-6e32-4a07-ac1f-85e98865d817 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:04.785974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78d77a-6e32-4a07-ac1f-85e98865d817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:04.815998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ad78d77a-6e32-4a07-ac1f-85e98865d817 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:04.817875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ad78d77a-6e32-4a07-ac1f-85e98865d817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:04.961609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 895/1785] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:04.984535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:04.986216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:05.138769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133daad7-bb03-474f-bf6d-bd980457ff03 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:05.143602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24973987-3f5c-47e7-b4ba-ae779e80af22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:05.144427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24973987-3f5c-47e7-b4ba-ae779e80af22 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:05.144828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24973987-3f5c-47e7-b4ba-ae779e80af22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:05.145208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24973987-3f5c-47e7-b4ba-ae779e80af22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:05.185287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24973987-3f5c-47e7-b4ba-ae779e80af22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:05.187126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24973987-3f5c-47e7-b4ba-ae779e80af22 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:05.203166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 891/1786] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:05.731099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78d77a-6e32-4a07-ac1f-85e98865d817 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:05.734156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ad78d77a-6e32-4a07-ac1f-85e98865d817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:05.806557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78d77a-6e32-4a07-ac1f-85e98865d817 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:05.838577 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:05.844613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:05.844775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:05.845021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] PATH_INFO: `/v3/projects/3c98c6ff51334e92b65da30bbae780a5` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:05.849911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c98c6ff51334e92b65da30bbae780a5)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:05.855808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:05.988266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:05.996108 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c98c6ff51334e92b65da30bbae780a5', 'request_id': 'req-5abedab9-7ad9-4963-91ff-163b8786c72a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:06.019624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c98c6ff51334e92b65da30bbae780a5', 'request_id': 'req-5abedab9-7ad9-4963-91ff-163b8786c72a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:06.032752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c98c6ff51334e92b65da30bbae780a5', 'request_id': 'req-5abedab9-7ad9-4963-91ff-163b8786c72a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:06.074764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] The token cache is being invalidate because project 3c98c6ff51334e92b65da30bbae780a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:06.075080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:06.076483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abedab9-7ad9-4963-91ff-163b8786c72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:06.082717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 896/1787] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:05 2022] DELETE /identity/v3/projects/3c98c6ff51334e92b65da30bbae780a5 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:06.083766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24973987-3f5c-47e7-b4ba-ae779e80af22 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:06.096308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24973987-3f5c-47e7-b4ba-ae779e80af22 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:06.188660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6054e-ca5b-4171-b27d-d004476415bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:06.190249 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6054e-ca5b-4171-b27d-d004476415bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:06.190854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6054e-ca5b-4171-b27d-d004476415bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:06.191293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6054e-ca5b-4171-b27d-d004476415bc admin admin] PATH_INFO: `/v3/users/6e2f759edf4a4b3491b8dbb31b078834` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:06.192733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6054e-ca5b-4171-b27d-d004476415bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2f759edf4a4b3491b8dbb31b078834)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:06.196130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6054e-ca5b-4171-b27d-d004476415bc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:06.280270 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36b6054e-ca5b-4171-b27d-d004476415bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2f759edf4a4b3491b8dbb31b078834'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:06.282321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36b6054e-ca5b-4171-b27d-d004476415bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2f759edf4a4b3491b8dbb31b078834'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:06.293900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36b6054e-ca5b-4171-b27d-d004476415bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2f759edf4a4b3491b8dbb31b078834'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:06.309824 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36b6054e-ca5b-4171-b27d-d004476415bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2f759edf4a4b3491b8dbb31b078834'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:06.311087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36b6054e-ca5b-4171-b27d-d004476415bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2f759edf4a4b3491b8dbb31b078834'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:06.330722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:06.347213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 892/1788] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:06 2022] DELETE /identity/v3/users/6e2f759edf4a4b3491b8dbb31b078834 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:06.857707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc969da3-9948-4916-b31d-8753f0a86e6a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:06.858303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc969da3-9948-4916-b31d-8753f0a86e6a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:06.858303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc969da3-9948-4916-b31d-8753f0a86e6a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:06.859599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 897/1789] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 06:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:31:06.880975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab01be8-70a3-4281-a96f-03c2a90ab2de None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:06.881471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab01be8-70a3-4281-a96f-03c2a90ab2de None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:06.882257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab01be8-70a3-4281-a96f-03c2a90ab2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:06.913293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1ab01be8-70a3-4281-a96f-03c2a90ab2de None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:06.915765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1ab01be8-70a3-4281-a96f-03c2a90ab2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:07.130231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 893/1790] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 06:31:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:07.200033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc969da3-9948-4916-b31d-8753f0a86e6a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:07.201807 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc969da3-9948-4916-b31d-8753f0a86e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:07.299202 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc969da3-9948-4916-b31d-8753f0a86e6a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:07.304994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:07.306824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:07.307480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:07.308241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] PATH_INFO: `/v3/projects/3fb45bcd0dfb4f2cbe8abd3af76b0a5e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:07.310325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fb45bcd0dfb4f2cbe8abd3af76b0a5e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:07.313614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:07.403081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fb45bcd0dfb4f2cbe8abd3af76b0a5e', 'request_id': 'req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:07.411471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fb45bcd0dfb4f2cbe8abd3af76b0a5e', 'request_id': 'req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:07.419916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fb45bcd0dfb4f2cbe8abd3af76b0a5e', 'request_id': 'req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:07.433167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] The token cache is being invalidate because project 3fb45bcd0dfb4f2cbe8abd3af76b0a5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:07.433454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:07.434187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:07.438025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 898/1791] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:07 2022] DELETE /identity/v3/projects/3fb45bcd0dfb4f2cbe8abd3af76b0a5e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:07.439982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab01be8-70a3-4281-a96f-03c2a90ab2de None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:07.452532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab01be8-70a3-4281-a96f-03c2a90ab2de None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:07.531513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:07.532454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:07.532735 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:07.533660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] PATH_INFO: `/v3/users/42264ff0039b4135958d6b00728552ec` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:07.535034 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] RBAC: Authorizing `identity:delete_user(user_id=42264ff0039b4135958d6b00728552ec)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:07.537184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:07.599703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42264ff0039b4135958d6b00728552ec'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:07.601573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42264ff0039b4135958d6b00728552ec'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:07.613624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42264ff0039b4135958d6b00728552ec'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:07.630613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42264ff0039b4135958d6b00728552ec'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:07.631304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42264ff0039b4135958d6b00728552ec'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:07.653347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 894/1792] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:07 2022] DELETE /identity/v3/users/42264ff0039b4135958d6b00728552ec => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:07.716979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:07.719055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:07.799897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:07.807814 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcece50e-73b6-4fb6-9eff-ae8ff4c67e19 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:07.889233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efd152-2218-494c-bfa9-0c6b4f10bd1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:07.890214 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efd152-2218-494c-bfa9-0c6b4f10bd1d service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:07.890519 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efd152-2218-494c-bfa9-0c6b4f10bd1d service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:07.890889 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efd152-2218-494c-bfa9-0c6b4f10bd1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:07.893079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-70efd152-2218-494c-bfa9-0c6b4f10bd1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:07.908954 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efd152-2218-494c-bfa9-0c6b4f10bd1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:07.910684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efd152-2218-494c-bfa9-0c6b4f10bd1d service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:07.923669 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 899/1793] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:31:08.326704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:08.329201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:08.349112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:08.447279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6d39b-085a-48a4-8312-45a4747d951d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:08.454437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:08.456455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:08.456906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:08.458021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] PATH_INFO: `/v3/projects/41c4a14a445a4fdc8f553e322cfdbb9b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:08.461677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41c4a14a445a4fdc8f553e322cfdbb9b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:08.463874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:08.520372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efd152-2218-494c-bfa9-0c6b4f10bd1d service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:08.522491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-70efd152-2218-494c-bfa9-0c6b4f10bd1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:08.550826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efd152-2218-494c-bfa9-0c6b4f10bd1d service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:08.558100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41c4a14a445a4fdc8f553e322cfdbb9b', 'request_id': 'req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.564731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41c4a14a445a4fdc8f553e322cfdbb9b', 'request_id': 'req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.573140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41c4a14a445a4fdc8f553e322cfdbb9b', 'request_id': 'req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.594615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] The token cache is being invalidate because project 41c4a14a445a4fdc8f553e322cfdbb9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:08.594957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.595589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.597665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 895/1794] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:08 2022] DELETE /identity/v3/projects/41c4a14a445a4fdc8f553e322cfdbb9b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:08.603455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:08.605453 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:08.639745 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:08.640837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:08.641685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:08.642140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] PATH_INFO: `/v3/projects/f1666fb7296945c4b3123dfc4bbfd568` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:08.643791 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1666fb7296945c4b3123dfc4bbfd568)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:08.645821 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:08.690745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee7d44-c901-40c2-acc3-1a5ebab0fed7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:08.696305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a9df4-3815-4d98-9c49-3cd480d3e838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:08.697493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a9df4-3815-4d98-9c49-3cd480d3e838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:08.697783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a9df4-3815-4d98-9c49-3cd480d3e838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:08.698106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a9df4-3815-4d98-9c49-3cd480d3e838 admin admin] PATH_INFO: `/v3/users/d33aad3725c645388274d4400a4e9ef0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:08.704020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a9df4-3815-4d98-9c49-3cd480d3e838 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d33aad3725c645388274d4400a4e9ef0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:08.706498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a9df4-3815-4d98-9c49-3cd480d3e838 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:08.758232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1666fb7296945c4b3123dfc4bbfd568', 'request_id': 'req-eda05d9d-2831-48bb-8890-7b85e8f171b4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.770117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1666fb7296945c4b3123dfc4bbfd568', 'request_id': 'req-eda05d9d-2831-48bb-8890-7b85e8f171b4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.781628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1666fb7296945c4b3123dfc4bbfd568', 'request_id': 'req-eda05d9d-2831-48bb-8890-7b85e8f171b4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.797074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] The token cache is being invalidate because project f1666fb7296945c4b3123dfc4bbfd568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:08.797535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.798126 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eda05d9d-2831-48bb-8890-7b85e8f171b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.799581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:08.806008 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 900/1795] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:08 2022] DELETE /identity/v3/projects/f1666fb7296945c4b3123dfc4bbfd568 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:08.807981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cec17-3682-48ee-b36f-97caa89f14e5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:08.808327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cec17-3682-48ee-b36f-97caa89f14e5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:08.808761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cec17-3682-48ee-b36f-97caa89f14e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:08.816391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-549a9df4-3815-4d98-9c49-3cd480d3e838 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd33aad3725c645388274d4400a4e9ef0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.817356 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-549a9df4-3815-4d98-9c49-3cd480d3e838 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd33aad3725c645388274d4400a4e9ef0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.828578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-549a9df4-3815-4d98-9c49-3cd480d3e838 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd33aad3725c645388274d4400a4e9ef0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.833016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8f1cec17-3682-48ee-b36f-97caa89f14e5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:08.834470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8f1cec17-3682-48ee-b36f-97caa89f14e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:08.850448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-549a9df4-3815-4d98-9c49-3cd480d3e838 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd33aad3725c645388274d4400a4e9ef0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.851733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-549a9df4-3815-4d98-9c49-3cd480d3e838 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd33aad3725c645388274d4400a4e9ef0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:08.852748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:08.874760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 896/1796] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:08 2022] DELETE /identity/v3/users/d33aad3725c645388274d4400a4e9ef0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:08.930875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 901/1797] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:08.940543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb039b-6289-4f01-a493-0e75617eccc9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:08.940907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb039b-6289-4f01-a493-0e75617eccc9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:08.941116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb039b-6289-4f01-a493-0e75617eccc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:08.972019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9acb039b-6289-4f01-a493-0e75617eccc9 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:08.973826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9acb039b-6289-4f01-a493-0e75617eccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:09.050945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 897/1798] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:09.062057 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cec17-3682-48ee-b36f-97caa89f14e5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.077505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cec17-3682-48ee-b36f-97caa89f14e5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.085115 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d1d52-0fe7-426e-b90e-4dc638485a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.090449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d1d52-0fe7-426e-b90e-4dc638485a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.091338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d1d52-0fe7-426e-b90e-4dc638485a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.091780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d1d52-0fe7-426e-b90e-4dc638485a62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.092741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d1d52-0fe7-426e-b90e-4dc638485a62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:09.093548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d1d52-0fe7-426e-b90e-4dc638485a62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.095812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d1d52-0fe7-426e-b90e-4dc638485a62 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:09.129852 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 902/1799] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:09.135161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb039b-6289-4f01-a493-0e75617eccc9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.147348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb039b-6289-4f01-a493-0e75617eccc9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.154998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212771ae-98de-4b90-9181-3849f4335d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.155486 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212771ae-98de-4b90-9181-3849f4335d6d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.156044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212771ae-98de-4b90-9181-3849f4335d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.156044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212771ae-98de-4b90-9181-3849f4335d6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.156744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212771ae-98de-4b90-9181-3849f4335d6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.159468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212771ae-98de-4b90-9181-3849f4335d6d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:09.198615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 898/1800] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:09 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:09.200273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d1d52-0fe7-426e-b90e-4dc638485a62 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.213601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d1d52-0fe7-426e-b90e-4dc638485a62 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.284061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212771ae-98de-4b90-9181-3849f4335d6d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.285882 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-212771ae-98de-4b90-9181-3849f4335d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:09.286157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0af25-179f-4883-a9f3-f0405b5931c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.287043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0af25-179f-4883-a9f3-f0405b5931c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.287288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0af25-179f-4883-a9f3-f0405b5931c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.287614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0af25-179f-4883-a9f3-f0405b5931c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.289723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0af25-179f-4883-a9f3-f0405b5931c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.291921 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0af25-179f-4883-a9f3-f0405b5931c8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:09.305466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212771ae-98de-4b90-9181-3849f4335d6d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.312642 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.313365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.313700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.314090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] PATH_INFO: `/v3/projects/e002fe23cfac48948fcb0de1f21e9dc8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.315372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e002fe23cfac48948fcb0de1f21e9dc8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.317417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:09.322283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 903/1801] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:09 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:09.324283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0af25-179f-4883-a9f3-f0405b5931c8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.334814 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0af25-179f-4883-a9f3-f0405b5931c8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.340608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadcff3-dee3-4c05-ba5c-64b62ddbee76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.342146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadcff3-dee3-4c05-ba5c-64b62ddbee76 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.342146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadcff3-dee3-4c05-ba5c-64b62ddbee76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.342356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadcff3-dee3-4c05-ba5c-64b62ddbee76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.342961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadcff3-dee3-4c05-ba5c-64b62ddbee76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.344587 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadcff3-dee3-4c05-ba5c-64b62ddbee76 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:09.356896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 904/1802] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:09.362360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadcff3-dee3-4c05-ba5c-64b62ddbee76 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.374429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadcff3-dee3-4c05-ba5c-64b62ddbee76 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.379146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e002fe23cfac48948fcb0de1f21e9dc8', 'request_id': 'req-f5a25931-667f-4853-88ed-52bc1a2443e8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:09.396282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e002fe23cfac48948fcb0de1f21e9dc8', 'request_id': 'req-f5a25931-667f-4853-88ed-52bc1a2443e8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:09.408766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e002fe23cfac48948fcb0de1f21e9dc8', 'request_id': 'req-f5a25931-667f-4853-88ed-52bc1a2443e8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:09.424702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] The token cache is being invalidate because project e002fe23cfac48948fcb0de1f21e9dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:09.425098 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:09.425664 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f5a25931-667f-4853-88ed-52bc1a2443e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:09.427286 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 899/1803] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:09 2022] DELETE /identity/v3/projects/e002fe23cfac48948fcb0de1f21e9dc8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:09.433625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0b399-3bc2-4434-8f52-6dd17d4808a4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.433920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0b399-3bc2-4434-8f52-6dd17d4808a4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.434224 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0b399-3bc2-4434-8f52-6dd17d4808a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.442992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0d034-e54d-42c9-8132-ae35195bb609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.443816 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0d034-e54d-42c9-8132-ae35195bb609 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.444115 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0d034-e54d-42c9-8132-ae35195bb609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.444485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0d034-e54d-42c9-8132-ae35195bb609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.445455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0d034-e54d-42c9-8132-ae35195bb609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.447188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0d034-e54d-42c9-8132-ae35195bb609 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:09.452349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7ec0b399-3bc2-4434-8f52-6dd17d4808a4 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:09.454477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7ec0b399-3bc2-4434-8f52-6dd17d4808a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:09.458765 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 905/1804] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:09.463983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0d034-e54d-42c9-8132-ae35195bb609 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.465626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fbc0d034-e54d-42c9-8132-ae35195bb609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:09.466082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 900/1805] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:09.474256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676737e-06a9-4a4d-b948-cd029107bbfa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.474545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676737e-06a9-4a4d-b948-cd029107bbfa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.474829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676737e-06a9-4a4d-b948-cd029107bbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.479373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0d034-e54d-42c9-8132-ae35195bb609 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.484527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8c762-acca-403a-aac9-c6d99263e5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.485932 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8c762-acca-403a-aac9-c6d99263e5b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.486321 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8c762-acca-403a-aac9-c6d99263e5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.486761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8c762-acca-403a-aac9-c6d99263e5b3 admin admin] PATH_INFO: `/v3/projects/7ac5962942fd41cd971d7310aab99e94/users/a15200db42bd44398feba0bea955a2d7/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.502452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7676737e-06a9-4a4d-b948-cd029107bbfa None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:09.503663 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7676737e-06a9-4a4d-b948-cd029107bbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:09.506050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8c762-acca-403a-aac9-c6d99263e5b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac5962942fd41cd971d7310aab99e94, user_id=a15200db42bd44398feba0bea955a2d7, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.509457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8c762-acca-403a-aac9-c6d99263e5b3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:09.512514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 901/1806] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:09.516422 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676737e-06a9-4a4d-b948-cd029107bbfa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.521102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:09.528640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676737e-06a9-4a4d-b948-cd029107bbfa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.539152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 906/1807] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:09 2022] PUT /identity/v3/projects/7ac5962942fd41cd971d7310aab99e94/users/a15200db42bd44398feba0bea955a2d7/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:09.550249 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:09.556906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73bb16-166c-4a75-b6e5-e56bea8b05ac None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.557222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73bb16-166c-4a75-b6e5-e56bea8b05ac None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.557513 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73bb16-166c-4a75-b6e5-e56bea8b05ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.614318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6e73bb16-166c-4a75-b6e5-e56bea8b05ac None None] MFA Rules not processed for user `a15200db42bd44398feba0bea955a2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:09.616404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6e73bb16-166c-4a75-b6e5-e56bea8b05ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:09.636440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e418b-56d1-428d-9efe-1dfce45e2b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.637252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e418b-56d1-428d-9efe-1dfce45e2b6c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.637601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e418b-56d1-428d-9efe-1dfce45e2b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.637978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e418b-56d1-428d-9efe-1dfce45e2b6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.638804 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e418b-56d1-428d-9efe-1dfce45e2b6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:09.639190 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e418b-56d1-428d-9efe-1dfce45e2b6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.641689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e418b-56d1-428d-9efe-1dfce45e2b6c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:09.663751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 902/1808] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:31:09.671094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e418b-56d1-428d-9efe-1dfce45e2b6c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.690063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e418b-56d1-428d-9efe-1dfce45e2b6c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.700925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5365e-d562-4b59-b362-04950fe09e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.701620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5365e-d562-4b59-b362-04950fe09e39 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.702775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5365e-d562-4b59-b362-04950fe09e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.703080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5365e-d562-4b59-b362-04950fe09e39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.703800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5365e-d562-4b59-b362-04950fe09e39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.706335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5365e-d562-4b59-b362-04950fe09e39 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:09.738410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 907/1809] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:09.743687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73bb16-166c-4a75-b6e5-e56bea8b05ac None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.756370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 903/1810] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:09 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:09.758093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5365e-d562-4b59-b362-04950fe09e39 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.760527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73bb16-166c-4a75-b6e5-e56bea8b05ac None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.837036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60503b0-c923-489c-8302-6d934ef9937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.838468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60503b0-c923-489c-8302-6d934ef9937d admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.838955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60503b0-c923-489c-8302-6d934ef9937d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.839421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60503b0-c923-489c-8302-6d934ef9937d admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.842049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60503b0-c923-489c-8302-6d934ef9937d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.844581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60503b0-c923-489c-8302-6d934ef9937d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:09.853833 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5365e-d562-4b59-b362-04950fe09e39 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.858895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7d59b-33eb-4c5f-8d0a-835a86eadf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.859916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7d59b-33eb-4c5f-8d0a-835a86eadf71 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.860391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7d59b-33eb-4c5f-8d0a-835a86eadf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.860761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7d59b-33eb-4c5f-8d0a-835a86eadf71 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.861529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7d59b-33eb-4c5f-8d0a-835a86eadf71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.863803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7d59b-33eb-4c5f-8d0a-835a86eadf71 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:09.887889 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 908/1811] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:09 2022] POST /identity/v3/projects => generated 369 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:31:09.889067 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60503b0-c923-489c-8302-6d934ef9937d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.902613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 904/1812] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:09 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:09.906794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7d59b-33eb-4c5f-8d0a-835a86eadf71 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:09.919498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7d59b-33eb-4c5f-8d0a-835a86eadf71 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.984902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60503b0-c923-489c-8302-6d934ef9937d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:09.991177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d03dda-c216-4aa5-95f8-c53e70195d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:09.992004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d03dda-c216-4aa5-95f8-c53e70195d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:09.992212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d03dda-c216-4aa5-95f8-c53e70195d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:09.992925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d03dda-c216-4aa5-95f8-c53e70195d9c admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:09.993194 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d03dda-c216-4aa5-95f8-c53e70195d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:09.995695 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d03dda-c216-4aa5-95f8-c53e70195d9c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:10.001818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dea220-12dd-4ac5-a2e1-2f395b124d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:10.003539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dea220-12dd-4ac5-a2e1-2f395b124d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.003539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dea220-12dd-4ac5-a2e1-2f395b124d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.003539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dea220-12dd-4ac5-a2e1-2f395b124d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.004077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dea220-12dd-4ac5-a2e1-2f395b124d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:10.007659 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dea220-12dd-4ac5-a2e1-2f395b124d6b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:10.020173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 905/1813] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:10.024117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:10.025620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dea220-12dd-4ac5-a2e1-2f395b124d6b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:10.033148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 909/1814] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:09 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:10.036790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d03dda-c216-4aa5-95f8-c53e70195d9c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:10.048993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:10.055252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dea220-12dd-4ac5-a2e1-2f395b124d6b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:10.062222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85602f59-997a-4b69-b56d-24570c18a8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:10.062985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85602f59-997a-4b69-b56d-24570c18a8ec admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.063503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85602f59-997a-4b69-b56d-24570c18a8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.063921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85602f59-997a-4b69-b56d-24570c18a8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.065623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85602f59-997a-4b69-b56d-24570c18a8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:10.065856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d03dda-c216-4aa5-95f8-c53e70195d9c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:10.069121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85602f59-997a-4b69-b56d-24570c18a8ec admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:10.077775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512dec2-dd78-4135-a41e-a0d26e2686f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:10.078667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512dec2-dd78-4135-a41e-a0d26e2686f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.078835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512dec2-dd78-4135-a41e-a0d26e2686f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.079140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512dec2-dd78-4135-a41e-a0d26e2686f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.079926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512dec2-dd78-4135-a41e-a0d26e2686f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:10.081416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512dec2-dd78-4135-a41e-a0d26e2686f4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:10.096093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 910/1815] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:10.097222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512dec2-dd78-4135-a41e-a0d26e2686f4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:10.100763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 906/1816] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:10.104622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85602f59-997a-4b69-b56d-24570c18a8ec admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:10.111574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512dec2-dd78-4135-a41e-a0d26e2686f4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:10.114558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85602f59-997a-4b69-b56d-24570c18a8ec admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:10.117826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139f040-de28-45fb-a762-d50520fa5e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:10.119373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8eb1c6-5a4f-4084-b264-b1da97019700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:10.119734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139f040-de28-45fb-a762-d50520fa5e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.120129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139f040-de28-45fb-a762-d50520fa5e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.120328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8eb1c6-5a4f-4084-b264-b1da97019700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.120596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139f040-de28-45fb-a762-d50520fa5e7c admin admin] PATH_INFO: `/v3/projects/19475d51b9854a3b8c87de4a9e057bdf/users/340dd674014d4142a34c1428d335e7a1/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.120690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8eb1c6-5a4f-4084-b264-b1da97019700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.120880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8eb1c6-5a4f-4084-b264-b1da97019700 admin admin] PATH_INFO: `/v3/projects/e612f06f547449ac9e9b95dcd32d5928/users/ca97ac5a572b4c52804a26f5e6b8320b/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.137832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8eb1c6-5a4f-4084-b264-b1da97019700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e612f06f547449ac9e9b95dcd32d5928, user_id=ca97ac5a572b4c52804a26f5e6b8320b, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:10.139737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139f040-de28-45fb-a762-d50520fa5e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=19475d51b9854a3b8c87de4a9e057bdf, user_id=340dd674014d4142a34c1428d335e7a1, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:10.140659 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8eb1c6-5a4f-4084-b264-b1da97019700 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:10.142672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139f040-de28-45fb-a762-d50520fa5e7c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:10.169153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 911/1817] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:10 2022] PUT /identity/v3/projects/19475d51b9854a3b8c87de4a9e057bdf/users/340dd674014d4142a34c1428d335e7a1/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:10.175186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 907/1818] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:10 2022] PUT /identity/v3/projects/e612f06f547449ac9e9b95dcd32d5928/users/ca97ac5a572b4c52804a26f5e6b8320b/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:10.175297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139f040-de28-45fb-a762-d50520fa5e7c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:10.177703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7a538-1a44-4af8-9c52-c57d939ba4d4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.178077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7a538-1a44-4af8-9c52-c57d939ba4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.178449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7a538-1a44-4af8-9c52-c57d939ba4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.242725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fed7a538-1a44-4af8-9c52-c57d939ba4d4 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:10.243954 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fed7a538-1a44-4af8-9c52-c57d939ba4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:10.294462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139f040-de28-45fb-a762-d50520fa5e7c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:10.300454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af27e6-b4e7-4d68-88b1-a84768aecc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:10.301485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af27e6-b4e7-4d68-88b1-a84768aecc55 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.302168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af27e6-b4e7-4d68-88b1-a84768aecc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.302667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af27e6-b4e7-4d68-88b1-a84768aecc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.304423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af27e6-b4e7-4d68-88b1-a84768aecc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:10.306445 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af27e6-b4e7-4d68-88b1-a84768aecc55 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:10.321570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 908/1819] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:10.331880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7a538-1a44-4af8-9c52-c57d939ba4d4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:10.333493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fed7a538-1a44-4af8-9c52-c57d939ba4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:10.334471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 912/1820] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:10.338529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af27e6-b4e7-4d68-88b1-a84768aecc55 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:10.355908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af27e6-b4e7-4d68-88b1-a84768aecc55 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:10.361713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8140c7-cfd8-4396-a9d2-154c317720e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:10.362809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8140c7-cfd8-4396-a9d2-154c317720e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.363104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8140c7-cfd8-4396-a9d2-154c317720e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.363386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8140c7-cfd8-4396-a9d2-154c317720e2 admin admin] PATH_INFO: `/v3/projects/19475d51b9854a3b8c87de4a9e057bdf/users/340dd674014d4142a34c1428d335e7a1/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.365974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8140c7-cfd8-4396-a9d2-154c317720e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19475d51b9854a3b8c87de4a9e057bdf, user_id=340dd674014d4142a34c1428d335e7a1, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:10.369913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8140c7-cfd8-4396-a9d2-154c317720e2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:10.381743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 913/1821] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:10 2022] PUT /identity/v3/projects/19475d51b9854a3b8c87de4a9e057bdf/users/340dd674014d4142a34c1428d335e7a1/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:10.393118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5070a1-ba50-4558-b3dc-a58994df8539 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.393412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5070a1-ba50-4558-b3dc-a58994df8539 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.393699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5070a1-ba50-4558-b3dc-a58994df8539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.417686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7a538-1a44-4af8-9c52-c57d939ba4d4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:10.425422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4e5070a1-ba50-4558-b3dc-a58994df8539 None None] MFA Rules not processed for user `340dd674014d4142a34c1428d335e7a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:10.427292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4e5070a1-ba50-4558-b3dc-a58994df8539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:10.487779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab10ab-33f9-4e2e-b2b4-1499747ac7ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:10.488844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab10ab-33f9-4e2e-b2b4-1499747ac7ce service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.489137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab10ab-33f9-4e2e-b2b4-1499747ac7ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.489721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab10ab-33f9-4e2e-b2b4-1499747ac7ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.517989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 914/1822] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:10 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:10.530517 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5070a1-ba50-4558-b3dc-a58994df8539 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:10.532092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4e5070a1-ba50-4558-b3dc-a58994df8539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:10.541435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:10.574550 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab10ab-33f9-4e2e-b2b4-1499747ac7ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:10.576404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:10.576549 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab10ab-33f9-4e2e-b2b4-1499747ac7ce service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:10.604541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 909/1823] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:31:10.614946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5070a1-ba50-4558-b3dc-a58994df8539 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:10.621252 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b41381-21cb-4ca9-a4ca-fee96555c589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:10.621978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41381-21cb-4ca9-a4ca-fee96555c589 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.622564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41381-21cb-4ca9-a4ca-fee96555c589 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.622858 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41381-21cb-4ca9-a4ca-fee96555c589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.634000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b41381-21cb-4ca9-a4ca-fee96555c589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:10.635609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b41381-21cb-4ca9-a4ca-fee96555c589 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:10.648020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 915/1824] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:10 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:31:10.697514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab10ab-33f9-4e2e-b2b4-1499747ac7ce service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:10.700399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f5ab10ab-33f9-4e2e-b2b4-1499747ac7ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:10.716540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab10ab-33f9-4e2e-b2b4-1499747ac7ce service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:10.720798 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f6c72-72fd-432e-a437-810b6a773701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:10.721541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f6c72-72fd-432e-a437-810b6a773701 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:10.721894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f6c72-72fd-432e-a437-810b6a773701 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:10.722299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f6c72-72fd-432e-a437-810b6a773701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:10.804868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f6c72-72fd-432e-a437-810b6a773701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:10.806579 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f6c72-72fd-432e-a437-810b6a773701 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:10.819107 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 910/1825] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:11.373545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fe022-d581-4196-8798-4cb8aba8e3dd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:11.374089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fe022-d581-4196-8798-4cb8aba8e3dd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:11.374168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fe022-d581-4196-8798-4cb8aba8e3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:11.402305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ad5fe022-d581-4196-8798-4cb8aba8e3dd None None] MFA Rules not processed for user `ca97ac5a572b4c52804a26f5e6b8320b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:11.403786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ad5fe022-d581-4196-8798-4cb8aba8e3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:11.476601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 916/1826] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:11.484624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f6c72-72fd-432e-a437-810b6a773701 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:11.493889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f6c72-72fd-432e-a437-810b6a773701 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:11.497810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf631b7-91fe-490f-9d90-b7dd47ccb19c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:11.498434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf631b7-91fe-490f-9d90-b7dd47ccb19c service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:11.498715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf631b7-91fe-490f-9d90-b7dd47ccb19c service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:11.498991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf631b7-91fe-490f-9d90-b7dd47ccb19c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:11.508662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf631b7-91fe-490f-9d90-b7dd47ccb19c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:11.510131 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf631b7-91fe-490f-9d90-b7dd47ccb19c service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:11.521382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 911/1827] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:13.935001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c5072-83fb-41c5-b57a-b1d32c610a4d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:13.935624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c5072-83fb-41c5-b57a-b1d32c610a4d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:13.935624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c5072-83fb-41c5-b57a-b1d32c610a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:13.958132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5d5c5072-83fb-41c5-b57a-b1d32c610a4d None None] MFA Rules not processed for user `ca97ac5a572b4c52804a26f5e6b8320b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:13.959388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5d5c5072-83fb-41c5-b57a-b1d32c610a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:13.968091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 917/1828] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:14.407865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf631b7-91fe-490f-9d90-b7dd47ccb19c service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:14.419853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf631b7-91fe-490f-9d90-b7dd47ccb19c service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:14.424314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:14.425187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:14.425467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:14.425742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] PATH_INFO: `/v3/users/a15200db42bd44398feba0bea955a2d7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:14.426978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a15200db42bd44398feba0bea955a2d7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:14.430203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:14.482314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a15200db42bd44398feba0bea955a2d7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:14.482994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a15200db42bd44398feba0bea955a2d7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:14.491843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a15200db42bd44398feba0bea955a2d7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:14.508722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a15200db42bd44398feba0bea955a2d7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:14.510033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a15200db42bd44398feba0bea955a2d7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:14.527333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 912/1829] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:14 2022] DELETE /identity/v3/users/a15200db42bd44398feba0bea955a2d7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:14.532163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8d5cc-4c4e-4a62-b219-0e574ca302d0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:14.532645 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8d5cc-4c4e-4a62-b219-0e574ca302d0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:14.533129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8d5cc-4c4e-4a62-b219-0e574ca302d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:14.552506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-82d8d5cc-4c4e-4a62-b219-0e574ca302d0 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:14.554337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-82d8d5cc-4c4e-4a62-b219-0e574ca302d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:14.631596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 918/1830] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:14.638965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:14.640399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:14.688504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:14.712761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9eeaa-8a22-4891-aa4d-8e5e3bd23273 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:14.716499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecd987-478c-4177-a6e1-d5d9b9359283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:14.717156 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecd987-478c-4177-a6e1-d5d9b9359283 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:14.717442 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecd987-478c-4177-a6e1-d5d9b9359283 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:14.717730 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecd987-478c-4177-a6e1-d5d9b9359283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:14.729138 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecd987-478c-4177-a6e1-d5d9b9359283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:14.730629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecd987-478c-4177-a6e1-d5d9b9359283 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:14.740923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 913/1831] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:15.089233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8d5cc-4c4e-4a62-b219-0e574ca302d0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:15.090947 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-82d8d5cc-4c4e-4a62-b219-0e574ca302d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:15.108795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8d5cc-4c4e-4a62-b219-0e574ca302d0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:15.116972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:15.118512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:15.118944 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:15.119396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] PATH_INFO: `/v3/projects/7ac5962942fd41cd971d7310aab99e94` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:15.121062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac5962942fd41cd971d7310aab99e94)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:15.123250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:15.173736 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac5962942fd41cd971d7310aab99e94', 'request_id': 'req-665f15a5-e042-4f5a-be34-d3fd9403c752'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.177541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:15.183997 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac5962942fd41cd971d7310aab99e94', 'request_id': 'req-665f15a5-e042-4f5a-be34-d3fd9403c752'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.192029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac5962942fd41cd971d7310aab99e94', 'request_id': 'req-665f15a5-e042-4f5a-be34-d3fd9403c752'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.204262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] The token cache is being invalidate because project 7ac5962942fd41cd971d7310aab99e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:15.204550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.205213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.206826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 919/1832] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:15 2022] DELETE /identity/v3/projects/7ac5962942fd41cd971d7310aab99e94 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:15.210734 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecd987-478c-4177-a6e1-d5d9b9359283 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:15.220540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecd987-478c-4177-a6e1-d5d9b9359283 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:15.296251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619b8bf-024a-4f18-bd8d-7600d35a6bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:15.297124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619b8bf-024a-4f18-bd8d-7600d35a6bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:15.297404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619b8bf-024a-4f18-bd8d-7600d35a6bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:15.297678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619b8bf-024a-4f18-bd8d-7600d35a6bf2 admin admin] PATH_INFO: `/v3/users/340dd674014d4142a34c1428d335e7a1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:15.298876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619b8bf-024a-4f18-bd8d-7600d35a6bf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=340dd674014d4142a34c1428d335e7a1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:15.300783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619b8bf-024a-4f18-bd8d-7600d35a6bf2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:15.345587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a619b8bf-024a-4f18-bd8d-7600d35a6bf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '340dd674014d4142a34c1428d335e7a1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.346200 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a619b8bf-024a-4f18-bd8d-7600d35a6bf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '340dd674014d4142a34c1428d335e7a1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.353688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a619b8bf-024a-4f18-bd8d-7600d35a6bf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '340dd674014d4142a34c1428d335e7a1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.365895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a619b8bf-024a-4f18-bd8d-7600d35a6bf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '340dd674014d4142a34c1428d335e7a1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.367256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a619b8bf-024a-4f18-bd8d-7600d35a6bf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '340dd674014d4142a34c1428d335e7a1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.379831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 914/1833] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:15 2022] DELETE /identity/v3/users/340dd674014d4142a34c1428d335e7a1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:15.649490 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:15.650932 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:15.667958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f15a5-e042-4f5a-be34-d3fd9403c752 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:15.735777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:15.736653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:15.736936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:15.737216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] PATH_INFO: `/v3/projects/19475d51b9854a3b8c87de4a9e057bdf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:15.738473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19475d51b9854a3b8c87de4a9e057bdf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:15.740441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:15.792971 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19475d51b9854a3b8c87de4a9e057bdf', 'request_id': 'req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.800255 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19475d51b9854a3b8c87de4a9e057bdf', 'request_id': 'req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.806811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19475d51b9854a3b8c87de4a9e057bdf', 'request_id': 'req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.817264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] The token cache is being invalidate because project 19475d51b9854a3b8c87de4a9e057bdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:15.817553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.818223 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f5d0316b-5d34-4c95-a1e9-0ed5bbce6b36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:15.819840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 920/1834] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:15 2022] DELETE /identity/v3/projects/19475d51b9854a3b8c87de4a9e057bdf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:15.824826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7a6b1-bfae-4312-b2e9-4c27edc87548 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:15.825274 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7a6b1-bfae-4312-b2e9-4c27edc87548 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:15.825362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7a6b1-bfae-4312-b2e9-4c27edc87548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:15.843887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-99e7a6b1-bfae-4312-b2e9-4c27edc87548 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:15.845084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-99e7a6b1-bfae-4312-b2e9-4c27edc87548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:15.893871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:15.907581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 915/1835] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:15.916325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ba9b-1f85-4a2a-a2f0-d78710308515 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:15.916624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ba9b-1f85-4a2a-a2f0-d78710308515 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:15.916913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ba9b-1f85-4a2a-a2f0-d78710308515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:15.935186 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d8a8ba9b-1f85-4a2a-a2f0-d78710308515 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:15.936384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d8a8ba9b-1f85-4a2a-a2f0-d78710308515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:15.944270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 921/1836] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:15.948533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7a6b1-bfae-4312-b2e9-4c27edc87548 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:15.958293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7a6b1-bfae-4312-b2e9-4c27edc87548 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:15.963651 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a2646-a62e-4a97-8aba-84a829d24324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:15.964320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a2646-a62e-4a97-8aba-84a829d24324 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:15.964605 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a2646-a62e-4a97-8aba-84a829d24324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:15.964889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a2646-a62e-4a97-8aba-84a829d24324 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:15.965624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a2646-a62e-4a97-8aba-84a829d24324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:15.965987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a2646-a62e-4a97-8aba-84a829d24324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:15.967628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a2646-a62e-4a97-8aba-84a829d24324 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:15.980152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 916/1837] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:15.984429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ba9b-1f85-4a2a-a2f0-d78710308515 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:15.995550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ba9b-1f85-4a2a-a2f0-d78710308515 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:16.001707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c623faa-3b93-4ec1-8463-90e97c8d06cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:16.002946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c623faa-3b93-4ec1-8463-90e97c8d06cc admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.002946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c623faa-3b93-4ec1-8463-90e97c8d06cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.003124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c623faa-3b93-4ec1-8463-90e97c8d06cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.003926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c623faa-3b93-4ec1-8463-90e97c8d06cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:16.005861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c623faa-3b93-4ec1-8463-90e97c8d06cc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:16.037061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 922/1838] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:15 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:16.041622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a2646-a62e-4a97-8aba-84a829d24324 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:16.052611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a2646-a62e-4a97-8aba-84a829d24324 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:16.118971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ad99a-1014-47a0-af92-5024a7c56de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:16.119648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad99a-1014-47a0-af92-5024a7c56de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.119930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad99a-1014-47a0-af92-5024a7c56de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.120201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad99a-1014-47a0-af92-5024a7c56de7 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.120933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ad99a-1014-47a0-af92-5024a7c56de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:16.122887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ad99a-1014-47a0-af92-5024a7c56de7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:16.144236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 917/1839] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:16 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:16.149140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c623faa-3b93-4ec1-8463-90e97c8d06cc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:16.160023 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c623faa-3b93-4ec1-8463-90e97c8d06cc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:16.165012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530a500-aa99-4742-a7b6-967ced781789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:16.165688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530a500-aa99-4742-a7b6-967ced781789 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.166005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530a500-aa99-4742-a7b6-967ced781789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.166379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530a500-aa99-4742-a7b6-967ced781789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.167022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530a500-aa99-4742-a7b6-967ced781789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:16.168626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530a500-aa99-4742-a7b6-967ced781789 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:16.179551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 923/1840] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:16.183557 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ad99a-1014-47a0-af92-5024a7c56de7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:16.195206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ad99a-1014-47a0-af92-5024a7c56de7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:16.204934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289892a-cf1d-4285-b3d9-ed8a1e858b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:16.206082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289892a-cf1d-4285-b3d9-ed8a1e858b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.206082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289892a-cf1d-4285-b3d9-ed8a1e858b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.206455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289892a-cf1d-4285-b3d9-ed8a1e858b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.207266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289892a-cf1d-4285-b3d9-ed8a1e858b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:16.209213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289892a-cf1d-4285-b3d9-ed8a1e858b60 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:16.224527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 918/1841] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:16.226490 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530a500-aa99-4742-a7b6-967ced781789 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:16.237591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530a500-aa99-4742-a7b6-967ced781789 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:16.246096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f98e-430d-4896-a157-1d241461c8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:16.247033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f98e-430d-4896-a157-1d241461c8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.247317 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f98e-430d-4896-a157-1d241461c8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.247594 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f98e-430d-4896-a157-1d241461c8f4 admin admin] PATH_INFO: `/v3/projects/91467c89ec5b4b0e9404073ddb62db00/users/c96a8f0ba29f4b87b0992424613107ff/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.268925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98f98e-430d-4896-a157-1d241461c8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91467c89ec5b4b0e9404073ddb62db00, user_id=c96a8f0ba29f4b87b0992424613107ff, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:16.271794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98f98e-430d-4896-a157-1d241461c8f4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:16.299367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 924/1842] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:16 2022] PUT /identity/v3/projects/91467c89ec5b4b0e9404073ddb62db00/users/c96a8f0ba29f4b87b0992424613107ff/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:16.311097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29ca2c-eab3-4446-ab53-f6b388f82cc3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.311437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29ca2c-eab3-4446-ab53-f6b388f82cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.311745 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29ca2c-eab3-4446-ab53-f6b388f82cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.341050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cd29ca2c-eab3-4446-ab53-f6b388f82cc3 None None] MFA Rules not processed for user `c96a8f0ba29f4b87b0992424613107ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:16.342529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd29ca2c-eab3-4446-ab53-f6b388f82cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:16.449287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 919/1843] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:16.454430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f98e-430d-4896-a157-1d241461c8f4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:16.471698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:16.552490 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f98e-430d-4896-a157-1d241461c8f4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:16.559348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a4099-4590-44d1-9a4e-6fc4ae47d15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:16.560730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a4099-4590-44d1-9a4e-6fc4ae47d15b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.560730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a4099-4590-44d1-9a4e-6fc4ae47d15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.560730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a4099-4590-44d1-9a4e-6fc4ae47d15b admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.561267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a4099-4590-44d1-9a4e-6fc4ae47d15b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:16.563416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a4099-4590-44d1-9a4e-6fc4ae47d15b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:16.596394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 925/1844] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:16 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:16.601874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29ca2c-eab3-4446-ab53-f6b388f82cc3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:16.613101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29ca2c-eab3-4446-ab53-f6b388f82cc3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:16.672131 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:16.684604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc970b8-c047-459a-b2f4-41f871707d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:16.685385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc970b8-c047-459a-b2f4-41f871707d15 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.685614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc970b8-c047-459a-b2f4-41f871707d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.685891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc970b8-c047-459a-b2f4-41f871707d15 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.686685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc970b8-c047-459a-b2f4-41f871707d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:16.688657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc970b8-c047-459a-b2f4-41f871707d15 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:16.720038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 920/1845] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:16 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:16.725331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a4099-4590-44d1-9a4e-6fc4ae47d15b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:16.738054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a4099-4590-44d1-9a4e-6fc4ae47d15b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:16.744066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4a9d-656d-4f16-be3d-6ac89cb6e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:16.744826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4a9d-656d-4f16-be3d-6ac89cb6e82a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.745118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4a9d-656d-4f16-be3d-6ac89cb6e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.745418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4a9d-656d-4f16-be3d-6ac89cb6e82a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.746283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e4a9d-656d-4f16-be3d-6ac89cb6e82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:16.748277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e4a9d-656d-4f16-be3d-6ac89cb6e82a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:16.764328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 926/1846] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:16.769407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc970b8-c047-459a-b2f4-41f871707d15 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:16.783724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc970b8-c047-459a-b2f4-41f871707d15 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:16.792805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26f661-c1cb-496b-90d9-ef05e13eb334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:16.793724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26f661-c1cb-496b-90d9-ef05e13eb334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.794267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26f661-c1cb-496b-90d9-ef05e13eb334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.794398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26f661-c1cb-496b-90d9-ef05e13eb334 admin admin] PATH_INFO: `/v3/projects/7fac3ded304a4c90b3fa43ee9bc16b40/users/4688f24de859417fa721ec81e467011c/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.813320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26f661-c1cb-496b-90d9-ef05e13eb334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fac3ded304a4c90b3fa43ee9bc16b40, user_id=4688f24de859417fa721ec81e467011c, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:16.816488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26f661-c1cb-496b-90d9-ef05e13eb334 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:16.836679 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 921/1847] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:16 2022] PUT /identity/v3/projects/7fac3ded304a4c90b3fa43ee9bc16b40/users/4688f24de859417fa721ec81e467011c/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:16.840831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4a9d-656d-4f16-be3d-6ac89cb6e82a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:16.852590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4a9d-656d-4f16-be3d-6ac89cb6e82a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:16.930063 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84a72b-cc46-4c7e-9713-66f7edf22457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:16.930833 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84a72b-cc46-4c7e-9713-66f7edf22457 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.931121 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84a72b-cc46-4c7e-9713-66f7edf22457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.931402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84a72b-cc46-4c7e-9713-66f7edf22457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.932949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84a72b-cc46-4c7e-9713-66f7edf22457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:16.936097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84a72b-cc46-4c7e-9713-66f7edf22457 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:16.951720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 927/1848] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:16.956154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26f661-c1cb-496b-90d9-ef05e13eb334 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:16.970395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26f661-c1cb-496b-90d9-ef05e13eb334 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:16.982958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a7592-ceb5-45dd-aa8d-d4de2775b0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:16.984013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a7592-ceb5-45dd-aa8d-d4de2775b0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:16.984404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a7592-ceb5-45dd-aa8d-d4de2775b0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:16.984751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a7592-ceb5-45dd-aa8d-d4de2775b0b7 admin admin] PATH_INFO: `/v3/projects/7fac3ded304a4c90b3fa43ee9bc16b40/users/4688f24de859417fa721ec81e467011c/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:16.987284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a7592-ceb5-45dd-aa8d-d4de2775b0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fac3ded304a4c90b3fa43ee9bc16b40, user_id=4688f24de859417fa721ec81e467011c, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:16.990607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a7592-ceb5-45dd-aa8d-d4de2775b0b7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:17.003012 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 922/1849] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:16 2022] PUT /identity/v3/projects/7fac3ded304a4c90b3fa43ee9bc16b40/users/4688f24de859417fa721ec81e467011c/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:17.016651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3830343-f4ed-4fd2-b492-cf3aa05711a9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:17.016943 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3830343-f4ed-4fd2-b492-cf3aa05711a9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:17.017281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3830343-f4ed-4fd2-b492-cf3aa05711a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:17.053585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a3830343-f4ed-4fd2-b492-cf3aa05711a9 None None] MFA Rules not processed for user `4688f24de859417fa721ec81e467011c`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:17.055025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a3830343-f4ed-4fd2-b492-cf3aa05711a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:17.136365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 928/1850] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:17.148345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a7592-ceb5-45dd-aa8d-d4de2775b0b7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:17.150454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-405a7592-ceb5-45dd-aa8d-d4de2775b0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:17.211109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a7592-ceb5-45dd-aa8d-d4de2775b0b7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:17.214850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec747b-e97e-4ff9-8a4a-5abc53e823b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:17.215505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec747b-e97e-4ff9-8a4a-5abc53e823b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:17.215792 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec747b-e97e-4ff9-8a4a-5abc53e823b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:17.216070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec747b-e97e-4ff9-8a4a-5abc53e823b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:17.226815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec747b-e97e-4ff9-8a4a-5abc53e823b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:17.228171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec747b-e97e-4ff9-8a4a-5abc53e823b4 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:17.249649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 923/1851] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:17.614328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3830343-f4ed-4fd2-b492-cf3aa05711a9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:17.620341 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:17.626434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3830343-f4ed-4fd2-b492-cf3aa05711a9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:17.632078 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb319a-b765-4963-8c5c-b60f8ffddc4f tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4688f24de859417fa721ec81e467011c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fac3ded304a4c90b3fa43ee9bc16b40', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:17.633486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb319a-b765-4963-8c5c-b60f8ffddc4f tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:17.634175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb319a-b765-4963-8c5c-b60f8ffddc4f tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:17.634471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb319a-b765-4963-8c5c-b60f8ffddc4f tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:17.635233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb319a-b765-4963-8c5c-b60f8ffddc4f tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:17.635550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb319a-b765-4963-8c5c-b60f8ffddc4f tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:17.637646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb319a-b765-4963-8c5c-b60f8ffddc4f tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:17.650684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 929/1852] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:17.654967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec747b-e97e-4ff9-8a4a-5abc53e823b4 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:17.666302 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec747b-e97e-4ff9-8a4a-5abc53e823b4 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:17.671066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37cbcd8-7e61-48e3-9c84-ad45c9bdd765 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4688f24de859417fa721ec81e467011c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fac3ded304a4c90b3fa43ee9bc16b40', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:17.671759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cbcd8-7e61-48e3-9c84-ad45c9bdd765 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:17.672052 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cbcd8-7e61-48e3-9c84-ad45c9bdd765 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:17.672331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cbcd8-7e61-48e3-9c84-ad45c9bdd765 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:17.673038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37cbcd8-7e61-48e3-9c84-ad45c9bdd765 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:17.675537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37cbcd8-7e61-48e3-9c84-ad45c9bdd765 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:17.706388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 924/1853] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:17 2022] POST /identity/v3/projects => generated 353 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:17.707614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb319a-b765-4963-8c5c-b60f8ffddc4f tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:17.718886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb319a-b765-4963-8c5c-b60f8ffddc4f tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:17.815561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61bba1-0194-4032-a19f-118b041e0ddc tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4688f24de859417fa721ec81e467011c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fac3ded304a4c90b3fa43ee9bc16b40', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:17.816255 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61bba1-0194-4032-a19f-118b041e0ddc tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:17.816555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61bba1-0194-4032-a19f-118b041e0ddc tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:17.817333 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61bba1-0194-4032-a19f-118b041e0ddc tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:17.817642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61bba1-0194-4032-a19f-118b041e0ddc tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:17.818007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61bba1-0194-4032-a19f-118b041e0ddc tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:17.820714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61bba1-0194-4032-a19f-118b041e0ddc tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:17.836861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 930/1854] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:17.888747 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37cbcd8-7e61-48e3-9c84-ad45c9bdd765 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:17.902048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37cbcd8-7e61-48e3-9c84-ad45c9bdd765 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:17.906970 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4688f24de859417fa721ec81e467011c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fac3ded304a4c90b3fa43ee9bc16b40', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:17.909409 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:17.909909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:17.910412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] PATH_INFO: `/v3/projects/0674c1ef703b4dc5a6ce94846b067f7a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:17.911806 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0674c1ef703b4dc5a6ce94846b067f7a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:17.914288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:17.968280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0674c1ef703b4dc5a6ce94846b067f7a', 'request_id': 'req-7cf9852f-c955-4746-bcd9-68ccb2b119d6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:17.974972 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0674c1ef703b4dc5a6ce94846b067f7a', 'request_id': 'req-7cf9852f-c955-4746-bcd9-68ccb2b119d6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:17.984141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0674c1ef703b4dc5a6ce94846b067f7a', 'request_id': 'req-7cf9852f-c955-4746-bcd9-68ccb2b119d6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:18.004912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] The token cache is being invalidate because project 0674c1ef703b4dc5a6ce94846b067f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:18.006690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:18.010427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7cf9852f-c955-4746-bcd9-68ccb2b119d6 tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:18.012655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 925/1855] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:17 2022] DELETE /identity/v3/projects/0674c1ef703b4dc5a6ce94846b067f7a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:18.337084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61bba1-0194-4032-a19f-118b041e0ddc tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:18.338845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ae61bba1-0194-4032-a19f-118b041e0ddc tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:18.407407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61bba1-0194-4032-a19f-118b041e0ddc tempest-VolumeQuotaClassesTest-1095494053 tempest-VolumeQuotaClassesTest-1095494053-project-admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:18.464332 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:18.465133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:18.465413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:18.465692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] PATH_INFO: `/v3/users/c96a8f0ba29f4b87b0992424613107ff` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:18.466917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96a8f0ba29f4b87b0992424613107ff)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:18.468851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:18.507906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:18.518827 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96a8f0ba29f4b87b0992424613107ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:18.532061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96a8f0ba29f4b87b0992424613107ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:18.541016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96a8f0ba29f4b87b0992424613107ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:18.541685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96a8f0ba29f4b87b0992424613107ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:18.542331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96a8f0ba29f4b87b0992424613107ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:18.563193 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 931/1856] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:18 2022] DELETE /identity/v3/users/c96a8f0ba29f4b87b0992424613107ff => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:18.568133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082e54c-acfd-4b5c-8bf4-dd0d61fd3b20 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:18.568519 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082e54c-acfd-4b5c-8bf4-dd0d61fd3b20 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:18.568868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082e54c-acfd-4b5c-8bf4-dd0d61fd3b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:18.589275 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:18.596688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1082e54c-acfd-4b5c-8bf4-dd0d61fd3b20 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:18.600056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1082e54c-acfd-4b5c-8bf4-dd0d61fd3b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:18.679201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 926/1857] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:18.689070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:18.690808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:18.767055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb9232-c5a0-4353-8301-309c4530c23f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:18.770842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0a77e-8c77-457e-b9eb-dc872fe21ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:18.771529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a77e-8c77-457e-b9eb-dc872fe21ada service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:18.771817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a77e-8c77-457e-b9eb-dc872fe21ada service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:18.772100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a77e-8c77-457e-b9eb-dc872fe21ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:18.782458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0a77e-8c77-457e-b9eb-dc872fe21ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:18.783977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0a77e-8c77-457e-b9eb-dc872fe21ada service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:18.794490 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 932/1858] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:19.118554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082e54c-acfd-4b5c-8bf4-dd0d61fd3b20 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:19.120102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1082e54c-acfd-4b5c-8bf4-dd0d61fd3b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:19.135098 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082e54c-acfd-4b5c-8bf4-dd0d61fd3b20 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:19.140400 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:19.141343 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:19.141703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:19.142079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] PATH_INFO: `/v3/projects/91467c89ec5b4b0e9404073ddb62db00` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:19.143459 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91467c89ec5b4b0e9404073ddb62db00)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:19.146357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:19.199183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91467c89ec5b4b0e9404073ddb62db00', 'request_id': 'req-fb7aa992-b0a2-4706-bf22-707675aca6c3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:19.204960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91467c89ec5b4b0e9404073ddb62db00', 'request_id': 'req-fb7aa992-b0a2-4706-bf22-707675aca6c3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:19.212076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91467c89ec5b4b0e9404073ddb62db00', 'request_id': 'req-fb7aa992-b0a2-4706-bf22-707675aca6c3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:19.225642 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] The token cache is being invalidate because project 91467c89ec5b4b0e9404073ddb62db00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:19.225966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:19.226918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:19.228499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 927/1859] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:19 2022] DELETE /identity/v3/projects/91467c89ec5b4b0e9404073ddb62db00 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:19.232451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0a77e-8c77-457e-b9eb-dc872fe21ada service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:19.242541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0a77e-8c77-457e-b9eb-dc872fe21ada service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:19.302011 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43e48f-a2de-4aad-97f6-1c6935536754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:19.302934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43e48f-a2de-4aad-97f6-1c6935536754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:19.303217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43e48f-a2de-4aad-97f6-1c6935536754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:19.304156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43e48f-a2de-4aad-97f6-1c6935536754 admin admin] PATH_INFO: `/v3/users/4688f24de859417fa721ec81e467011c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:19.304831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43e48f-a2de-4aad-97f6-1c6935536754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4688f24de859417fa721ec81e467011c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:19.306817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43e48f-a2de-4aad-97f6-1c6935536754 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:19.370457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c43e48f-a2de-4aad-97f6-1c6935536754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4688f24de859417fa721ec81e467011c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:19.382057 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:19.389676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c43e48f-a2de-4aad-97f6-1c6935536754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4688f24de859417fa721ec81e467011c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:19.406533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c43e48f-a2de-4aad-97f6-1c6935536754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4688f24de859417fa721ec81e467011c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:19.408478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c43e48f-a2de-4aad-97f6-1c6935536754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4688f24de859417fa721ec81e467011c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:19.409389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c43e48f-a2de-4aad-97f6-1c6935536754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4688f24de859417fa721ec81e467011c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:19.436837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 933/1860] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:19 2022] DELETE /identity/v3/users/4688f24de859417fa721ec81e467011c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:19.963928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:19.965787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:19.981574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa992-b0a2-4706-bf22-707675aca6c3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:20.083540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:20.084612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:20.084938 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:20.085270 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] PATH_INFO: `/v3/projects/7fac3ded304a4c90b3fa43ee9bc16b40` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:20.086930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fac3ded304a4c90b3fa43ee9bc16b40)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:20.088921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:20.131033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:20.148927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fac3ded304a4c90b3fa43ee9bc16b40', 'request_id': 'req-faf6ac9a-cbd1-4dba-991b-595b2b467606'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:20.159636 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fac3ded304a4c90b3fa43ee9bc16b40', 'request_id': 'req-faf6ac9a-cbd1-4dba-991b-595b2b467606'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:20.172080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fac3ded304a4c90b3fa43ee9bc16b40', 'request_id': 'req-faf6ac9a-cbd1-4dba-991b-595b2b467606'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:20.198281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] The token cache is being invalidate because project 7fac3ded304a4c90b3fa43ee9bc16b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:20.198609 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:20.199247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-faf6ac9a-cbd1-4dba-991b-595b2b467606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:20.202460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 928/1861] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:19 2022] DELETE /identity/v3/projects/7fac3ded304a4c90b3fa43ee9bc16b40 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:20.206438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da7b39-157a-48ec-8900-694a0d5ea7ce None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:20.207087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da7b39-157a-48ec-8900-694a0d5ea7ce None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:20.207087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da7b39-157a-48ec-8900-694a0d5ea7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:20.238546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-92da7b39-157a-48ec-8900-694a0d5ea7ce None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:20.240020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-92da7b39-157a-48ec-8900-694a0d5ea7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:20.307227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 934/1862] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:20.316442 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b15f7-c3ca-4e8b-9e63-5b98b1da9247 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:20.316738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b15f7-c3ca-4e8b-9e63-5b98b1da9247 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:20.317025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b15f7-c3ca-4e8b-9e63-5b98b1da9247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:20.336085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ae1b15f7-c3ca-4e8b-9e63-5b98b1da9247 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:20.337354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ae1b15f7-c3ca-4e8b-9e63-5b98b1da9247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:20.345250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 929/1863] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:20.349309 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da7b39-157a-48ec-8900-694a0d5ea7ce None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:20.359612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da7b39-157a-48ec-8900-694a0d5ea7ce None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:20.365447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832ebc1-45ab-4a2d-bbbf-7a5f007359df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:20.365447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ebc1-45ab-4a2d-bbbf-7a5f007359df admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:20.365447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ebc1-45ab-4a2d-bbbf-7a5f007359df admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:20.365447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ebc1-45ab-4a2d-bbbf-7a5f007359df admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:20.366226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832ebc1-45ab-4a2d-bbbf-7a5f007359df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:20.366539 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832ebc1-45ab-4a2d-bbbf-7a5f007359df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:20.368129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832ebc1-45ab-4a2d-bbbf-7a5f007359df admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:20.380079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 935/1864] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:20.384938 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b15f7-c3ca-4e8b-9e63-5b98b1da9247 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:20.394239 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b15f7-c3ca-4e8b-9e63-5b98b1da9247 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:20.400332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bee890-83c5-46eb-931b-0e743d301c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:20.401246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bee890-83c5-46eb-931b-0e743d301c13 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:20.401525 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bee890-83c5-46eb-931b-0e743d301c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:20.401800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bee890-83c5-46eb-931b-0e743d301c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:20.402527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bee890-83c5-46eb-931b-0e743d301c13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:20.404434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bee890-83c5-46eb-931b-0e743d301c13 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:20.431596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 930/1865] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:20 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:20.435674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832ebc1-45ab-4a2d-bbbf-7a5f007359df admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:20.446064 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832ebc1-45ab-4a2d-bbbf-7a5f007359df admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:20.504722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611cb89-e546-4444-a41a-a414497fda0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:20.505410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611cb89-e546-4444-a41a-a414497fda0c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:20.505694 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611cb89-e546-4444-a41a-a414497fda0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:20.506022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611cb89-e546-4444-a41a-a414497fda0c admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:20.507303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611cb89-e546-4444-a41a-a414497fda0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:20.509213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611cb89-e546-4444-a41a-a414497fda0c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:20.533525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 936/1866] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:20 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:20.534836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bee890-83c5-46eb-931b-0e743d301c13 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:20.545649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bee890-83c5-46eb-931b-0e743d301c13 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:20.550375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ba4c4-dec1-4dd8-9d12-2c0b011a8859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:20.551046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ba4c4-dec1-4dd8-9d12-2c0b011a8859 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:20.551329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ba4c4-dec1-4dd8-9d12-2c0b011a8859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:20.551613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ba4c4-dec1-4dd8-9d12-2c0b011a8859 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:20.552348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ba4c4-dec1-4dd8-9d12-2c0b011a8859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:20.554622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ba4c4-dec1-4dd8-9d12-2c0b011a8859 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:20.565518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 931/1867] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:20.570068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611cb89-e546-4444-a41a-a414497fda0c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:20.580338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611cb89-e546-4444-a41a-a414497fda0c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:20.585618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3832b-866a-4219-94e7-42d14614cb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:20.586275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3832b-866a-4219-94e7-42d14614cb78 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:20.586567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3832b-866a-4219-94e7-42d14614cb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:20.586866 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3832b-866a-4219-94e7-42d14614cb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:20.587604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3832b-866a-4219-94e7-42d14614cb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:20.589278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3832b-866a-4219-94e7-42d14614cb78 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:20.599791 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 937/1868] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:20.604384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ba4c4-dec1-4dd8-9d12-2c0b011a8859 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:20.614793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ba4c4-dec1-4dd8-9d12-2c0b011a8859 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:20.619671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00a302-0354-4375-928a-9679d4048c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:20.620737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00a302-0354-4375-928a-9679d4048c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:20.621130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00a302-0354-4375-928a-9679d4048c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:20.621284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00a302-0354-4375-928a-9679d4048c09 admin admin] PATH_INFO: `/v3/projects/85634a2f13044e8ca1deebae88534c81/users/62f16f13da144583ad6e885dc3c12300/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:20.638925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00a302-0354-4375-928a-9679d4048c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85634a2f13044e8ca1deebae88534c81, user_id=62f16f13da144583ad6e885dc3c12300, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:20.642797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00a302-0354-4375-928a-9679d4048c09 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:20.677206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 932/1869] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:20 2022] PUT /identity/v3/projects/85634a2f13044e8ca1deebae88534c81/users/62f16f13da144583ad6e885dc3c12300/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:20.690923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a939065-a5c4-4c86-8c52-c8c7c9211896 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:20.691581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a939065-a5c4-4c86-8c52-c8c7c9211896 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:20.692322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a939065-a5c4-4c86-8c52-c8c7c9211896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:20.724626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1a939065-a5c4-4c86-8c52-c8c7c9211896 None None] MFA Rules not processed for user `62f16f13da144583ad6e885dc3c12300`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:20.726248 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1a939065-a5c4-4c86-8c52-c8c7c9211896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:20.758401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:20.826803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 938/1870] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:20.844827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00a302-0354-4375-928a-9679d4048c09 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:20.848263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ef00a302-0354-4375-928a-9679d4048c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:20.922970 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00a302-0354-4375-928a-9679d4048c09 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:20.926540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb9e25-7e34-4e9c-8151-0703b195da8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:20.927187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb9e25-7e34-4e9c-8151-0703b195da8a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:20.927468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb9e25-7e34-4e9c-8151-0703b195da8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:20.927750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb9e25-7e34-4e9c-8151-0703b195da8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:20.938029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb9e25-7e34-4e9c-8151-0703b195da8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:20.939506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb9e25-7e34-4e9c-8151-0703b195da8a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:20.949263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 933/1871] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:20.970867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a939065-a5c4-4c86-8c52-c8c7c9211896 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:21.042488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a939065-a5c4-4c86-8c52-c8c7c9211896 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:21.056934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:21.056934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:21.056934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:21.056934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] PATH_INFO: `/v3/users/62f16f13da144583ad6e885dc3c12300` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:21.058560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62f16f13da144583ad6e885dc3c12300)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:21.063553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:21.165615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62f16f13da144583ad6e885dc3c12300'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:21.181689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62f16f13da144583ad6e885dc3c12300'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:21.193775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62f16f13da144583ad6e885dc3c12300'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:21.195002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62f16f13da144583ad6e885dc3c12300'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:21.199269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62f16f13da144583ad6e885dc3c12300'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:21.226585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 939/1872] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:20 2022] DELETE /identity/v3/users/62f16f13da144583ad6e885dc3c12300 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:21.231614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ac17b-ffe0-48e3-ac0f-c7ac5337099b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:21.232018 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ac17b-ffe0-48e3-ac0f-c7ac5337099b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:21.232248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ac17b-ffe0-48e3-ac0f-c7ac5337099b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:21.262522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-732ac17b-ffe0-48e3-ac0f-c7ac5337099b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:21.263947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-732ac17b-ffe0-48e3-ac0f-c7ac5337099b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:21.326656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:21.335799 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 934/1873] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:21.345134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:21.346597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:21.411850 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:21.417743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed9c7d-d42e-43ed-87ee-35147e34ec47 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:21.426872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67371edc-4589-4189-b727-fde16abfd871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:21.427680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67371edc-4589-4189-b727-fde16abfd871 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:21.427977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67371edc-4589-4189-b727-fde16abfd871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:21.428262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67371edc-4589-4189-b727-fde16abfd871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:21.444151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67371edc-4589-4189-b727-fde16abfd871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:21.445600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67371edc-4589-4189-b727-fde16abfd871 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:21.457324 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 940/1874] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:21.980858 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ac17b-ffe0-48e3-ac0f-c7ac5337099b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:21.982722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-732ac17b-ffe0-48e3-ac0f-c7ac5337099b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:21.994999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ac17b-ffe0-48e3-ac0f-c7ac5337099b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:22.000023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:22.000890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:22.001194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:22.001471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] PATH_INFO: `/v3/projects/85634a2f13044e8ca1deebae88534c81` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:22.002666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] RBAC: Authorizing `identity:delete_project(project_id=85634a2f13044e8ca1deebae88534c81)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:22.004567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:22.089514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85634a2f13044e8ca1deebae88534c81', 'request_id': 'req-d4565313-fbf1-45af-9f96-f78020e9524f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:22.095822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85634a2f13044e8ca1deebae88534c81', 'request_id': 'req-d4565313-fbf1-45af-9f96-f78020e9524f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:22.104843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85634a2f13044e8ca1deebae88534c81', 'request_id': 'req-d4565313-fbf1-45af-9f96-f78020e9524f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:22.122941 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] The token cache is being invalidate because project 85634a2f13044e8ca1deebae88534c81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:22.123333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:22.124066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4565313-fbf1-45af-9f96-f78020e9524f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:22.129398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 935/1875] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:21 2022] DELETE /identity/v3/projects/85634a2f13044e8ca1deebae88534c81 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:22.130939 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15952eba-e5e8-475d-a450-3c3d0c204003 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:22.131236 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15952eba-e5e8-475d-a450-3c3d0c204003 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:22.131554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15952eba-e5e8-475d-a450-3c3d0c204003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:22.153209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-15952eba-e5e8-475d-a450-3c3d0c204003 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:22.155425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-15952eba-e5e8-475d-a450-3c3d0c204003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:22.217640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 941/1876] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:22.225911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eb341-f6b5-4b28-aaea-f1f06d2ef1fc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:22.226795 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eb341-f6b5-4b28-aaea-f1f06d2ef1fc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:22.227076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eb341-f6b5-4b28-aaea-f1f06d2ef1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:22.245776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9b0eb341-f6b5-4b28-aaea-f1f06d2ef1fc None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:22.247000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9b0eb341-f6b5-4b28-aaea-f1f06d2ef1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:22.254727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 936/1877] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:22.258979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15952eba-e5e8-475d-a450-3c3d0c204003 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:22.269430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15952eba-e5e8-475d-a450-3c3d0c204003 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:22.274431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fdcf1-9b30-4c6c-80c3-f7d8f052ef0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:22.275139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fdcf1-9b30-4c6c-80c3-f7d8f052ef0e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:22.275493 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fdcf1-9b30-4c6c-80c3-f7d8f052ef0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:22.276033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fdcf1-9b30-4c6c-80c3-f7d8f052ef0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:22.276764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fdcf1-9b30-4c6c-80c3-f7d8f052ef0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:22.277076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fdcf1-9b30-4c6c-80c3-f7d8f052ef0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:22.278852 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fdcf1-9b30-4c6c-80c3-f7d8f052ef0e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:22.291426 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 942/1878] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:22.295427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0eb341-f6b5-4b28-aaea-f1f06d2ef1fc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:22.306875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0eb341-f6b5-4b28-aaea-f1f06d2ef1fc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:22.311901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9329ed5-20eb-47df-a955-ba987784dcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:22.312732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9329ed5-20eb-47df-a955-ba987784dcb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:22.313289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9329ed5-20eb-47df-a955-ba987784dcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:22.313711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9329ed5-20eb-47df-a955-ba987784dcb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:22.314621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9329ed5-20eb-47df-a955-ba987784dcb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:22.316669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9329ed5-20eb-47df-a955-ba987784dcb0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:22.348617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 937/1879] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:22 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:22.349732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fdcf1-9b30-4c6c-80c3-f7d8f052ef0e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:22.364524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fdcf1-9b30-4c6c-80c3-f7d8f052ef0e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:22.426165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ca893-7cce-4b11-a04b-7db3e6425cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:22.426884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ca893-7cce-4b11-a04b-7db3e6425cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:22.427171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ca893-7cce-4b11-a04b-7db3e6425cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:22.427449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ca893-7cce-4b11-a04b-7db3e6425cb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:22.428152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ca893-7cce-4b11-a04b-7db3e6425cb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:22.430278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ca893-7cce-4b11-a04b-7db3e6425cb1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:22.454592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 943/1880] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:22 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:31:22.458855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9329ed5-20eb-47df-a955-ba987784dcb0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:22.474316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9329ed5-20eb-47df-a955-ba987784dcb0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:22.482241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7f366-f330-42b1-ad5b-1c5aca4ad611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:22.483122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7f366-f330-42b1-ad5b-1c5aca4ad611 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:22.483655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7f366-f330-42b1-ad5b-1c5aca4ad611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:22.484109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7f366-f330-42b1-ad5b-1c5aca4ad611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:22.485323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7f366-f330-42b1-ad5b-1c5aca4ad611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:22.487321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7f366-f330-42b1-ad5b-1c5aca4ad611 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:22.505640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 938/1881] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:22.507683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ca893-7cce-4b11-a04b-7db3e6425cb1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:22.520912 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ca893-7cce-4b11-a04b-7db3e6425cb1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:22.529197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627892a-0b79-4731-a0db-bf1f3f13eeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:22.531689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627892a-0b79-4731-a0db-bf1f3f13eeb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:22.531689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627892a-0b79-4731-a0db-bf1f3f13eeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:22.531689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627892a-0b79-4731-a0db-bf1f3f13eeb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:22.532864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627892a-0b79-4731-a0db-bf1f3f13eeb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:22.535778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627892a-0b79-4731-a0db-bf1f3f13eeb0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:22.555474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 944/1882] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:22.557339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7f366-f330-42b1-ad5b-1c5aca4ad611 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:22.573461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7f366-f330-42b1-ad5b-1c5aca4ad611 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:22.582476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033156a-fb8f-49ee-83e1-2c2c0be73ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:22.585381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033156a-fb8f-49ee-83e1-2c2c0be73ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:22.585381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033156a-fb8f-49ee-83e1-2c2c0be73ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:22.586698 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033156a-fb8f-49ee-83e1-2c2c0be73ad5 admin admin] PATH_INFO: `/v3/projects/621128a29f214578a4ff5494a218ff23/users/434b2c730c9242edb365636e8f5a7bce/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:22.642370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033156a-fb8f-49ee-83e1-2c2c0be73ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=621128a29f214578a4ff5494a218ff23, user_id=434b2c730c9242edb365636e8f5a7bce, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:22.654712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033156a-fb8f-49ee-83e1-2c2c0be73ad5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:22.687915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 939/1883] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:22 2022] PUT /identity/v3/projects/621128a29f214578a4ff5494a218ff23/users/434b2c730c9242edb365636e8f5a7bce/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:22.697978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85a4a-db74-4377-b900-d2c86a129bdf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:22.698431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85a4a-db74-4377-b900-d2c86a129bdf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:22.698655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85a4a-db74-4377-b900-d2c86a129bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:22.730199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-46e85a4a-db74-4377-b900-d2c86a129bdf None None] MFA Rules not processed for user `434b2c730c9242edb365636e8f5a7bce`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:22.731424 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-46e85a4a-db74-4377-b900-d2c86a129bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:22.778197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:22.799687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 945/1884] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:22.811316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033156a-fb8f-49ee-83e1-2c2c0be73ad5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:22.814430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7033156a-fb8f-49ee-83e1-2c2c0be73ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:22.914054 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033156a-fb8f-49ee-83e1-2c2c0be73ad5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:22.917715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb13239-eb72-4c85-94a8-4ced04a75f43 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa2c8be3105a415ead65579dc18ba40d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:22.918399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb13239-eb72-4c85-94a8-4ced04a75f43 service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:22.918681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb13239-eb72-4c85-94a8-4ced04a75f43 service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:22.918958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb13239-eb72-4c85-94a8-4ced04a75f43 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:22.928062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb13239-eb72-4c85-94a8-4ced04a75f43 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:22.930397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb13239-eb72-4c85-94a8-4ced04a75f43 service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:22.940453 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 940/1885] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:29.041756 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e85a4a-db74-4377-b900-d2c86a129bdf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:29.047343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-46e85a4a-db74-4377-b900-d2c86a129bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:29.160862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e85a4a-db74-4377-b900-d2c86a129bdf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:29.166724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:29.167647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:29.168179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:29.168179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] PATH_INFO: `/v3/users/cc8c8b8b46dc4e4d91a415611f82ba23` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:29.169760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc8c8b8b46dc4e4d91a415611f82ba23)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:29.172959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:29.242976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc8c8b8b46dc4e4d91a415611f82ba23'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:29.281244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc8c8b8b46dc4e4d91a415611f82ba23'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:29.304798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc8c8b8b46dc4e4d91a415611f82ba23'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:29.305872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc8c8b8b46dc4e4d91a415611f82ba23'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:29.307123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc8c8b8b46dc4e4d91a415611f82ba23'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:29.328280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 946/1886] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:29 2022] DELETE /identity/v3/users/cc8c8b8b46dc4e4d91a415611f82ba23 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:29.332955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675a31f-ae2d-487a-ba0e-2dd88ac41bf5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:29.334885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675a31f-ae2d-487a-ba0e-2dd88ac41bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:29.335448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675a31f-ae2d-487a-ba0e-2dd88ac41bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:29.349803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:29.376250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7675a31f-ae2d-487a-ba0e-2dd88ac41bf5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:29.382076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7675a31f-ae2d-487a-ba0e-2dd88ac41bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:29.467843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 941/1887] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:29.478145 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:29.480257 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:29.545571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8ec52-936f-43f6-a5a2-50116fbd1407 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:29.549264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70707afa-ae09-4a7d-aeac-4f80c3282b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:29.549963 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70707afa-ae09-4a7d-aeac-4f80c3282b18 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:29.551095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70707afa-ae09-4a7d-aeac-4f80c3282b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:29.551392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70707afa-ae09-4a7d-aeac-4f80c3282b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:29.564225 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70707afa-ae09-4a7d-aeac-4f80c3282b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:29.565685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70707afa-ae09-4a7d-aeac-4f80c3282b18 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:29.578972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 947/1888] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:30.124852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675a31f-ae2d-487a-ba0e-2dd88ac41bf5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:30.126908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7675a31f-ae2d-487a-ba0e-2dd88ac41bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:30.140900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675a31f-ae2d-487a-ba0e-2dd88ac41bf5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:30.146510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:30.147628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:30.148011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:30.148432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] PATH_INFO: `/v3/projects/510a883c711246fd85309a230b68e97e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:30.149894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510a883c711246fd85309a230b68e97e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:30.152587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:30.229461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510a883c711246fd85309a230b68e97e', 'request_id': 'req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:30.251877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510a883c711246fd85309a230b68e97e', 'request_id': 'req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:30.262904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510a883c711246fd85309a230b68e97e', 'request_id': 'req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:30.289334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] The token cache is being invalidate because project 510a883c711246fd85309a230b68e97e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:30.290081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:30.291288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:30.295509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 942/1889] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:30 2022] DELETE /identity/v3/projects/510a883c711246fd85309a230b68e97e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:30.297148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70707afa-ae09-4a7d-aeac-4f80c3282b18 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:30.318778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70707afa-ae09-4a7d-aeac-4f80c3282b18 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:30.325338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:30.394538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5b1e0-bfb8-4839-8cf9-d6e45dec3e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:30.395811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5b1e0-bfb8-4839-8cf9-d6e45dec3e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:30.396350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5b1e0-bfb8-4839-8cf9-d6e45dec3e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:30.396795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5b1e0-bfb8-4839-8cf9-d6e45dec3e92 admin admin] PATH_INFO: `/v3/users/130aab7797274965be30463a72e267b3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:30.398289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5b1e0-bfb8-4839-8cf9-d6e45dec3e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=130aab7797274965be30463a72e267b3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:30.401022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5b1e0-bfb8-4839-8cf9-d6e45dec3e92 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:30.455346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-09b5b1e0-bfb8-4839-8cf9-d6e45dec3e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '130aab7797274965be30463a72e267b3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:30.472097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-09b5b1e0-bfb8-4839-8cf9-d6e45dec3e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '130aab7797274965be30463a72e267b3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:30.490082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-09b5b1e0-bfb8-4839-8cf9-d6e45dec3e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '130aab7797274965be30463a72e267b3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:30.492051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-09b5b1e0-bfb8-4839-8cf9-d6e45dec3e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '130aab7797274965be30463a72e267b3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:30.492954 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-09b5b1e0-bfb8-4839-8cf9-d6e45dec3e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '130aab7797274965be30463a72e267b3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:30.509179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 948/1890] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:30 2022] DELETE /identity/v3/users/130aab7797274965be30463a72e267b3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:30.935951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:30.938096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:30.952875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c42c7-9b85-4ce0-81e1-ea6bf5774a22 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:31.018238 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:31.019123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:31.019412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:31.019697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] PATH_INFO: `/v3/projects/0593b967f37d4186ae62959fea0ba9b5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:31.021193 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0593b967f37d4186ae62959fea0ba9b5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:31.023684 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:31.071424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:31.096977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0593b967f37d4186ae62959fea0ba9b5', 'request_id': 'req-d0259bda-1192-4747-b8d5-676a41ff696c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:31.103339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0593b967f37d4186ae62959fea0ba9b5', 'request_id': 'req-d0259bda-1192-4747-b8d5-676a41ff696c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:31.120871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0593b967f37d4186ae62959fea0ba9b5', 'request_id': 'req-d0259bda-1192-4747-b8d5-676a41ff696c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:31.138280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] The token cache is being invalidate because project 0593b967f37d4186ae62959fea0ba9b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:31.138675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:31.139394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d0259bda-1192-4747-b8d5-676a41ff696c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:31.141284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 943/1891] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:30 2022] DELETE /identity/v3/projects/0593b967f37d4186ae62959fea0ba9b5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:31.145923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73decf-79f0-4e62-bd56-e9be112d463c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:31.146314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73decf-79f0-4e62-bd56-e9be112d463c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:31.146648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73decf-79f0-4e62-bd56-e9be112d463c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:31.168537 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5d73decf-79f0-4e62-bd56-e9be112d463c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:31.170173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5d73decf-79f0-4e62-bd56-e9be112d463c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:31.236690 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 949/1892] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:31.246818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09abaf-fb44-40ce-8774-a93429e178f1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:31.247419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09abaf-fb44-40ce-8774-a93429e178f1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:31.247886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09abaf-fb44-40ce-8774-a93429e178f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:31.267812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8a09abaf-fb44-40ce-8774-a93429e178f1 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:31.269047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8a09abaf-fb44-40ce-8774-a93429e178f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:31.281902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 944/1893] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:31.283705 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73decf-79f0-4e62-bd56-e9be112d463c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:31.298788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73decf-79f0-4e62-bd56-e9be112d463c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:31.305871 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21f0b5-d5fa-41c8-86aa-d35b6942fb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:31.306718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21f0b5-d5fa-41c8-86aa-d35b6942fb7f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:31.307010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21f0b5-d5fa-41c8-86aa-d35b6942fb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:31.307291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21f0b5-d5fa-41c8-86aa-d35b6942fb7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:31.308088 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21f0b5-d5fa-41c8-86aa-d35b6942fb7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:31.308407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21f0b5-d5fa-41c8-86aa-d35b6942fb7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:31.310071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21f0b5-d5fa-41c8-86aa-d35b6942fb7f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:31.331014 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 950/1894] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:31.332688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09abaf-fb44-40ce-8774-a93429e178f1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:31.346562 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09abaf-fb44-40ce-8774-a93429e178f1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:31.353438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76991d2b-07b9-4cdf-94b6-0f3597d5ccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:31.354253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991d2b-07b9-4cdf-94b6-0f3597d5ccd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:31.354608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991d2b-07b9-4cdf-94b6-0f3597d5ccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:31.354963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991d2b-07b9-4cdf-94b6-0f3597d5ccd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:31.355750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76991d2b-07b9-4cdf-94b6-0f3597d5ccd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:31.358086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76991d2b-07b9-4cdf-94b6-0f3597d5ccd4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:31.429477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 945/1895] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:31 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:31.432423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21f0b5-d5fa-41c8-86aa-d35b6942fb7f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:31.447735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21f0b5-d5fa-41c8-86aa-d35b6942fb7f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:31.482482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:31.537246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c061a1-d5dd-4dae-b214-1ca148c6d21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:31.538596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c061a1-d5dd-4dae-b214-1ca148c6d21d admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:31.538877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c061a1-d5dd-4dae-b214-1ca148c6d21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:31.539157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c061a1-d5dd-4dae-b214-1ca148c6d21d admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:31.539884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c061a1-d5dd-4dae-b214-1ca148c6d21d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:31.541846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c061a1-d5dd-4dae-b214-1ca148c6d21d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:31.566151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 951/1896] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:31 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:31.571358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76991d2b-07b9-4cdf-94b6-0f3597d5ccd4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:31.584850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76991d2b-07b9-4cdf-94b6-0f3597d5ccd4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:31.590448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e89c6e-ec50-4d2c-9887-b59f5f9ba87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:31.591683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e89c6e-ec50-4d2c-9887-b59f5f9ba87b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:31.591683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e89c6e-ec50-4d2c-9887-b59f5f9ba87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:31.591897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e89c6e-ec50-4d2c-9887-b59f5f9ba87b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:31.592700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e89c6e-ec50-4d2c-9887-b59f5f9ba87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:31.594635 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e89c6e-ec50-4d2c-9887-b59f5f9ba87b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:31.608949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 946/1897] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:31.613085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c061a1-d5dd-4dae-b214-1ca148c6d21d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:31.623713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c061a1-d5dd-4dae-b214-1ca148c6d21d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:31.630020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfaea4-19fb-40bf-9269-a23229febcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:31.632733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfaea4-19fb-40bf-9269-a23229febcf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:31.632733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfaea4-19fb-40bf-9269-a23229febcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:31.633039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfaea4-19fb-40bf-9269-a23229febcf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:31.637782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfaea4-19fb-40bf-9269-a23229febcf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:31.637782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfaea4-19fb-40bf-9269-a23229febcf5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:31.650252 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 952/1898] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:31.652869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e89c6e-ec50-4d2c-9887-b59f5f9ba87b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:31.663978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e89c6e-ec50-4d2c-9887-b59f5f9ba87b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:31.669931 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca4d3b-cc95-4342-90e8-dbc15a0752b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:31.670946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca4d3b-cc95-4342-90e8-dbc15a0752b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:31.671305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca4d3b-cc95-4342-90e8-dbc15a0752b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:31.671661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca4d3b-cc95-4342-90e8-dbc15a0752b8 admin admin] PATH_INFO: `/v3/projects/eccc192e899c4dddb39d64d99301815a/users/e6f01d5c3c2749b3a40e0fbb91031701/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:31.699391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca4d3b-cc95-4342-90e8-dbc15a0752b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eccc192e899c4dddb39d64d99301815a, user_id=e6f01d5c3c2749b3a40e0fbb91031701, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:31.703383 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca4d3b-cc95-4342-90e8-dbc15a0752b8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:31.724778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 947/1899] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:31 2022] PUT /identity/v3/projects/eccc192e899c4dddb39d64d99301815a/users/e6f01d5c3c2749b3a40e0fbb91031701/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:31.735881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fe02c-6ebc-4dad-9825-00bcb50c73d3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:31.736639 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fe02c-6ebc-4dad-9825-00bcb50c73d3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:31.737018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fe02c-6ebc-4dad-9825-00bcb50c73d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:31.775283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d54fe02c-6ebc-4dad-9825-00bcb50c73d3 None None] MFA Rules not processed for user `e6f01d5c3c2749b3a40e0fbb91031701`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:31.777527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d54fe02c-6ebc-4dad-9825-00bcb50c73d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:31.906371 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 953/1900] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:31.907370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca4d3b-cc95-4342-90e8-dbc15a0752b8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:31.981133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca4d3b-cc95-4342-90e8-dbc15a0752b8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:31.989810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75937b4b-0413-4d44-8e17-4ef30b9af7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:31.991083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75937b4b-0413-4d44-8e17-4ef30b9af7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:31.991083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75937b4b-0413-4d44-8e17-4ef30b9af7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:31.991744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75937b4b-0413-4d44-8e17-4ef30b9af7f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:31.991906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75937b4b-0413-4d44-8e17-4ef30b9af7f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:31.994177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75937b4b-0413-4d44-8e17-4ef30b9af7f9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:32.029480 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 948/1901] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:31 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:32.031753 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fe02c-6ebc-4dad-9825-00bcb50c73d3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:32.046883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fe02c-6ebc-4dad-9825-00bcb50c73d3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:32.148386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6440c-bf16-41f7-a64f-aa3d25bc7072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:32.149080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6440c-bf16-41f7-a64f-aa3d25bc7072 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:32.149367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6440c-bf16-41f7-a64f-aa3d25bc7072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:32.149650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6440c-bf16-41f7-a64f-aa3d25bc7072 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:32.150783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6440c-bf16-41f7-a64f-aa3d25bc7072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:32.159946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6440c-bf16-41f7-a64f-aa3d25bc7072 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:32.193005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 954/1902] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:32 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:32.197253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75937b4b-0413-4d44-8e17-4ef30b9af7f9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:32.211859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75937b4b-0413-4d44-8e17-4ef30b9af7f9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:32.233029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53615821-9667-49e0-b234-6abee3ea99d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:32.233149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53615821-9667-49e0-b234-6abee3ea99d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:32.233390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53615821-9667-49e0-b234-6abee3ea99d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:32.233679 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53615821-9667-49e0-b234-6abee3ea99d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:32.234500 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53615821-9667-49e0-b234-6abee3ea99d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:32.236351 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53615821-9667-49e0-b234-6abee3ea99d6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:32.253007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 949/1903] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:32.258271 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6440c-bf16-41f7-a64f-aa3d25bc7072 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:32.271685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6440c-bf16-41f7-a64f-aa3d25bc7072 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:32.281511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5d620-5a7a-4524-9a64-71dd6bd59c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:32.282478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5d620-5a7a-4524-9a64-71dd6bd59c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:32.282775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5d620-5a7a-4524-9a64-71dd6bd59c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:32.283205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5d620-5a7a-4524-9a64-71dd6bd59c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:32.284195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5d620-5a7a-4524-9a64-71dd6bd59c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:32.286210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5d620-5a7a-4524-9a64-71dd6bd59c2f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:32.311449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 955/1904] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:32.316766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53615821-9667-49e0-b234-6abee3ea99d6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:32.348639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53615821-9667-49e0-b234-6abee3ea99d6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:32.360929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6e7ca-e928-4ce3-8f33-ffebe0211d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:32.362560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6e7ca-e928-4ce3-8f33-ffebe0211d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:32.363012 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6e7ca-e928-4ce3-8f33-ffebe0211d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:32.363447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6e7ca-e928-4ce3-8f33-ffebe0211d9b admin admin] PATH_INFO: `/v3/projects/aa0ced3fa61148f38376411f7e4a3819/users/3f5a0286bfa945afb5a675ea1c0037b6/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:32.399365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6e7ca-e928-4ce3-8f33-ffebe0211d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0ced3fa61148f38376411f7e4a3819, user_id=3f5a0286bfa945afb5a675ea1c0037b6, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:32.402753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6e7ca-e928-4ce3-8f33-ffebe0211d9b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:32.439704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 950/1905] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:32 2022] PUT /identity/v3/projects/aa0ced3fa61148f38376411f7e4a3819/users/3f5a0286bfa945afb5a675ea1c0037b6/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:32.450803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8b82f-cd33-4346-b532-457c5285370b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:32.451093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8b82f-cd33-4346-b532-457c5285370b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:32.451433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8b82f-cd33-4346-b532-457c5285370b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:32.458398 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:32.541636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b0b8b82f-cd33-4346-b532-457c5285370b None None] MFA Rules not processed for user `3f5a0286bfa945afb5a675ea1c0037b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:32.544790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b0b8b82f-cd33-4346-b532-457c5285370b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:32.730738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 956/1906] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:32.735454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6e7ca-e928-4ce3-8f33-ffebe0211d9b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:32.740330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:32.875796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6e7ca-e928-4ce3-8f33-ffebe0211d9b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:32.882851 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768edcce-193e-4bde-97e8-77a731da391d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:32.883559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768edcce-193e-4bde-97e8-77a731da391d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:32.884039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768edcce-193e-4bde-97e8-77a731da391d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:32.884329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768edcce-193e-4bde-97e8-77a731da391d admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:32.885055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768edcce-193e-4bde-97e8-77a731da391d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:32.887163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768edcce-193e-4bde-97e8-77a731da391d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:32.969026 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 951/1907] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:32 2022] POST /identity/v3/projects => generated 379 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:32.973118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8b82f-cd33-4346-b532-457c5285370b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:32.984992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8b82f-cd33-4346-b532-457c5285370b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:33.105618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408a006-da21-4839-aa76-8925b3d5f1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:33.107567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a006-da21-4839-aa76-8925b3d5f1be admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:33.108049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a006-da21-4839-aa76-8925b3d5f1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:33.108643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a006-da21-4839-aa76-8925b3d5f1be admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:33.109683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408a006-da21-4839-aa76-8925b3d5f1be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:33.112690 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408a006-da21-4839-aa76-8925b3d5f1be admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:33.169403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 957/1908] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:32 2022] POST /identity/v3/users => generated 296 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:33.172874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768edcce-193e-4bde-97e8-77a731da391d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:33.186070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768edcce-193e-4bde-97e8-77a731da391d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:33.192628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2ae46-f13b-46e5-9378-2ae6be8b5def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:33.199870 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2ae46-f13b-46e5-9378-2ae6be8b5def admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:33.200470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2ae46-f13b-46e5-9378-2ae6be8b5def admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:33.200959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2ae46-f13b-46e5-9378-2ae6be8b5def admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:33.201897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2ae46-f13b-46e5-9378-2ae6be8b5def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:33.203727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2ae46-f13b-46e5-9378-2ae6be8b5def admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:33.231129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408a006-da21-4839-aa76-8925b3d5f1be admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:33.238308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 952/1909] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:33.243768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408a006-da21-4839-aa76-8925b3d5f1be admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:33.249511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04186e3-69fa-49ce-b506-a91150989036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:33.250475 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04186e3-69fa-49ce-b506-a91150989036 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:33.250764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04186e3-69fa-49ce-b506-a91150989036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:33.251232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04186e3-69fa-49ce-b506-a91150989036 admin admin] PATH_INFO: `/v3/projects/1490106ca5b54949a4f1605e1264878c/users/e6bf19c917364c3195fdb2804418cef4/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:33.292936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04186e3-69fa-49ce-b506-a91150989036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1490106ca5b54949a4f1605e1264878c, user_id=e6bf19c917364c3195fdb2804418cef4, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:33.296958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04186e3-69fa-49ce-b506-a91150989036 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:33.324105 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 958/1910] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:33 2022] PUT /identity/v3/projects/1490106ca5b54949a4f1605e1264878c/users/e6bf19c917364c3195fdb2804418cef4/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:33.328279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2ae46-f13b-46e5-9378-2ae6be8b5def admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:33.352027 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2ae46-f13b-46e5-9378-2ae6be8b5def admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:33.459079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64a15a-2d9e-4644-a991-e4e799f721bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:33.459905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64a15a-2d9e-4644-a991-e4e799f721bf admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:33.460195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64a15a-2d9e-4644-a991-e4e799f721bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:33.460478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64a15a-2d9e-4644-a991-e4e799f721bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:33.461230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64a15a-2d9e-4644-a991-e4e799f721bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:33.463049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64a15a-2d9e-4644-a991-e4e799f721bf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:33.481529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 953/1911] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:33.486542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04186e3-69fa-49ce-b506-a91150989036 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:33.516704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04186e3-69fa-49ce-b506-a91150989036 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:33.522803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882ef90-4e49-4cd5-ac8b-cf9307319f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:33.524187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882ef90-4e49-4cd5-ac8b-cf9307319f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:33.524713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882ef90-4e49-4cd5-ac8b-cf9307319f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:33.525491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882ef90-4e49-4cd5-ac8b-cf9307319f87 admin admin] PATH_INFO: `/v3/projects/1490106ca5b54949a4f1605e1264878c/users/e6bf19c917364c3195fdb2804418cef4/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:33.527990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882ef90-4e49-4cd5-ac8b-cf9307319f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1490106ca5b54949a4f1605e1264878c, user_id=e6bf19c917364c3195fdb2804418cef4, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:33.531579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882ef90-4e49-4cd5-ac8b-cf9307319f87 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:33.554289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 959/1912] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:33 2022] PUT /identity/v3/projects/1490106ca5b54949a4f1605e1264878c/users/e6bf19c917364c3195fdb2804418cef4/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:33.562171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb128669-28fc-4572-91c5-569e73665683 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:33.563090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb128669-28fc-4572-91c5-569e73665683 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:33.563385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb128669-28fc-4572-91c5-569e73665683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:33.616420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bb128669-28fc-4572-91c5-569e73665683 None None] MFA Rules not processed for user `e6bf19c917364c3195fdb2804418cef4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:33.617993 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bb128669-28fc-4572-91c5-569e73665683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:33.695871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:33.704199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 954/1913] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:33.718208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882ef90-4e49-4cd5-ac8b-cf9307319f87 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:33.720317 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3882ef90-4e49-4cd5-ac8b-cf9307319f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:33.783460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:33.798300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882ef90-4e49-4cd5-ac8b-cf9307319f87 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:33.802076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43aa01-bfe4-4e17-8899-94e187961bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:33.803315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43aa01-bfe4-4e17-8899-94e187961bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:33.803649 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43aa01-bfe4-4e17-8899-94e187961bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:33.803934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43aa01-bfe4-4e17-8899-94e187961bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:33.814634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43aa01-bfe4-4e17-8899-94e187961bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:33.816173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43aa01-bfe4-4e17-8899-94e187961bb8 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:33.831389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 960/1914] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:33.875304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb128669-28fc-4572-91c5-569e73665683 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:33.877118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bb128669-28fc-4572-91c5-569e73665683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:33.890633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb128669-28fc-4572-91c5-569e73665683 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:33.894973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2ddde-9548-4926-a299-5d1e5f612c6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:33.895646 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ddde-9548-4926-a299-5d1e5f612c6a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:33.895934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ddde-9548-4926-a299-5d1e5f612c6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:33.896216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ddde-9548-4926-a299-5d1e5f612c6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:33.966435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2ddde-9548-4926-a299-5d1e5f612c6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:33.967922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2ddde-9548-4926-a299-5d1e5f612c6a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:33.978561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 955/1915] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:36.965897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43aa01-bfe4-4e17-8899-94e187961bb8 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:37.032152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43aa01-bfe4-4e17-8899-94e187961bb8 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:37.036815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:37.037731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:37.038045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:37.038335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] PATH_INFO: `/v3/users/e6f01d5c3c2749b3a40e0fbb91031701` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:37.039560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f01d5c3c2749b3a40e0fbb91031701)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:37.041525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:37.089653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f01d5c3c2749b3a40e0fbb91031701'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.101581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f01d5c3c2749b3a40e0fbb91031701'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.109953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f01d5c3c2749b3a40e0fbb91031701'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.110618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f01d5c3c2749b3a40e0fbb91031701'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.111394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f01d5c3c2749b3a40e0fbb91031701'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.125340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 961/1916] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:36 2022] DELETE /identity/v3/users/e6f01d5c3c2749b3a40e0fbb91031701 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:37.132009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6dac0-f798-4380-bcaf-4f92dcc5490e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:37.132492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6dac0-f798-4380-bcaf-4f92dcc5490e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:37.132492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6dac0-f798-4380-bcaf-4f92dcc5490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:37.150688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c9b6dac0-f798-4380-bcaf-4f92dcc5490e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:37.151966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c9b6dac0-f798-4380-bcaf-4f92dcc5490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:37.221974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 956/1917] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:37.232543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:37.234042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:37.298386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff8ce0-d7c4-4534-b8c3-93599f350820 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:37.302060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12a204-fb41-488d-a1f8-bae6681a1aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:37.302718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12a204-fb41-488d-a1f8-bae6681a1aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:37.302999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12a204-fb41-488d-a1f8-bae6681a1aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:37.304469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12a204-fb41-488d-a1f8-bae6681a1aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:37.313809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12a204-fb41-488d-a1f8-bae6681a1aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:37.315110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12a204-fb41-488d-a1f8-bae6681a1aa6 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:37.325247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 962/1918] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:37.695656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6dac0-f798-4380-bcaf-4f92dcc5490e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:37.697504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c9b6dac0-f798-4380-bcaf-4f92dcc5490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:37.718931 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6dac0-f798-4380-bcaf-4f92dcc5490e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:37.723543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:37.724417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:37.724700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:37.725688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] PATH_INFO: `/v3/projects/eccc192e899c4dddb39d64d99301815a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:37.727496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eccc192e899c4dddb39d64d99301815a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:37.729405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:37.778938 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eccc192e899c4dddb39d64d99301815a', 'request_id': 'req-fe165672-1ba3-4cb1-865e-054ddcc40052'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.784509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eccc192e899c4dddb39d64d99301815a', 'request_id': 'req-fe165672-1ba3-4cb1-865e-054ddcc40052'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.792217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eccc192e899c4dddb39d64d99301815a', 'request_id': 'req-fe165672-1ba3-4cb1-865e-054ddcc40052'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.806933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] The token cache is being invalidate because project eccc192e899c4dddb39d64d99301815a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:37.807271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.807907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.812518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 957/1919] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:37 2022] DELETE /identity/v3/projects/eccc192e899c4dddb39d64d99301815a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:37.813779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12a204-fb41-488d-a1f8-bae6681a1aa6 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:37.824395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12a204-fb41-488d-a1f8-bae6681a1aa6 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:37.829680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:37.893600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:37.894538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:37.894828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:37.895109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] PATH_INFO: `/v3/users/3f5a0286bfa945afb5a675ea1c0037b6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:37.896306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f5a0286bfa945afb5a675ea1c0037b6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:37.898998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:37.948222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f5a0286bfa945afb5a675ea1c0037b6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.963556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f5a0286bfa945afb5a675ea1c0037b6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.970302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f5a0286bfa945afb5a675ea1c0037b6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.971043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f5a0286bfa945afb5a675ea1c0037b6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.971663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f5a0286bfa945afb5a675ea1c0037b6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:37.985395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 963/1920] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:37 2022] DELETE /identity/v3/users/3f5a0286bfa945afb5a675ea1c0037b6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:38.734256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:38.736587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:38.755216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe165672-1ba3-4cb1-865e-054ddcc40052 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:38.784918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:38.859410 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:38.860879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:38.861326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:38.861775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] PATH_INFO: `/v3/projects/aa0ced3fa61148f38376411f7e4a3819` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:38.863986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa0ced3fa61148f38376411f7e4a3819)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:38.868310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:38.948939 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa0ced3fa61148f38376411f7e4a3819', 'request_id': 'req-1414cf6f-136a-4389-968d-8910d84078b4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:38.962221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa0ced3fa61148f38376411f7e4a3819', 'request_id': 'req-1414cf6f-136a-4389-968d-8910d84078b4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:38.969334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa0ced3fa61148f38376411f7e4a3819', 'request_id': 'req-1414cf6f-136a-4389-968d-8910d84078b4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:38.984308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] The token cache is being invalidate because project aa0ced3fa61148f38376411f7e4a3819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:38.984760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:38.985406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:38.987116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 958/1921] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:38 2022] DELETE /identity/v3/projects/aa0ced3fa61148f38376411f7e4a3819 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:38.991262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:38.992708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:39.008621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d983d-0ebb-4528-8582-d049b23a1c56 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:39.074124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3aae0e-138e-49cc-bdf1-1bb7d78e6997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:39.075024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aae0e-138e-49cc-bdf1-1bb7d78e6997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:39.075311 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aae0e-138e-49cc-bdf1-1bb7d78e6997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:39.075592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aae0e-138e-49cc-bdf1-1bb7d78e6997 admin admin] PATH_INFO: `/v3/users/e6bf19c917364c3195fdb2804418cef4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:39.076885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3aae0e-138e-49cc-bdf1-1bb7d78e6997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6bf19c917364c3195fdb2804418cef4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:39.079369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3aae0e-138e-49cc-bdf1-1bb7d78e6997 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:39.112829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:39.128075 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ec3aae0e-138e-49cc-bdf1-1bb7d78e6997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6bf19c917364c3195fdb2804418cef4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:39.140976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ec3aae0e-138e-49cc-bdf1-1bb7d78e6997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6bf19c917364c3195fdb2804418cef4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:39.148350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ec3aae0e-138e-49cc-bdf1-1bb7d78e6997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6bf19c917364c3195fdb2804418cef4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:39.149075 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ec3aae0e-138e-49cc-bdf1-1bb7d78e6997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6bf19c917364c3195fdb2804418cef4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:39.149680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ec3aae0e-138e-49cc-bdf1-1bb7d78e6997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6bf19c917364c3195fdb2804418cef4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:39.163743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 964/1922] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:38 2022] DELETE /identity/v3/users/e6bf19c917364c3195fdb2804418cef4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:39.435815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:39.437639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:39.451987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414cf6f-136a-4389-968d-8910d84078b4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:39.514999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:39.515797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:39.516083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:39.516363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] PATH_INFO: `/v3/projects/1490106ca5b54949a4f1605e1264878c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:39.517563 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1490106ca5b54949a4f1605e1264878c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:39.519531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:39.570330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1490106ca5b54949a4f1605e1264878c', 'request_id': 'req-b01034da-42c5-41d1-83eb-b1f99d0e881a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:39.572817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:39.577753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1490106ca5b54949a4f1605e1264878c', 'request_id': 'req-b01034da-42c5-41d1-83eb-b1f99d0e881a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:39.586145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1490106ca5b54949a4f1605e1264878c', 'request_id': 'req-b01034da-42c5-41d1-83eb-b1f99d0e881a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:39.599164 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] The token cache is being invalidate because project 1490106ca5b54949a4f1605e1264878c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:39.599521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:39.600822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b01034da-42c5-41d1-83eb-b1f99d0e881a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:39.603549 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 959/1923] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:39 2022] DELETE /identity/v3/projects/1490106ca5b54949a4f1605e1264878c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:39.609641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f401418-9840-4645-beb6-ba8d14a0b2ed None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:39.609953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f401418-9840-4645-beb6-ba8d14a0b2ed None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:39.610250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f401418-9840-4645-beb6-ba8d14a0b2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:39.633843 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7f401418-9840-4645-beb6-ba8d14a0b2ed None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:39.635194 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7f401418-9840-4645-beb6-ba8d14a0b2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:39.719373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 965/1924] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:39.727590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9723dc2-3827-4ab5-a733-ab106d7b4c3d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:39.727889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9723dc2-3827-4ab5-a733-ab106d7b4c3d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:39.728174 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9723dc2-3827-4ab5-a733-ab106d7b4c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:39.747737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e9723dc2-3827-4ab5-a733-ab106d7b4c3d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:39.749189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e9723dc2-3827-4ab5-a733-ab106d7b4c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:39.764554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 960/1925] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:39.768912 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f401418-9840-4645-beb6-ba8d14a0b2ed None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:39.779837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f401418-9840-4645-beb6-ba8d14a0b2ed None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:39.784480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5b093-6fc9-450c-ad58-3f2445f5cee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:39.785142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5b093-6fc9-450c-ad58-3f2445f5cee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:39.785429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5b093-6fc9-450c-ad58-3f2445f5cee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:39.785770 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5b093-6fc9-450c-ad58-3f2445f5cee8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:39.786650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5b093-6fc9-450c-ad58-3f2445f5cee8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:39.786965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5b093-6fc9-450c-ad58-3f2445f5cee8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:39.788775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5b093-6fc9-450c-ad58-3f2445f5cee8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:39.804497 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 966/1926] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:39.807934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9723dc2-3827-4ab5-a733-ab106d7b4c3d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:39.817247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9723dc2-3827-4ab5-a733-ab106d7b4c3d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:39.822509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fb16f-f6fe-4765-b322-959d0a8220c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:39.823204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fb16f-f6fe-4765-b322-959d0a8220c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:39.823489 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fb16f-f6fe-4765-b322-959d0a8220c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:39.823771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fb16f-f6fe-4765-b322-959d0a8220c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:39.824462 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fb16f-f6fe-4765-b322-959d0a8220c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:39.826398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fb16f-f6fe-4765-b322-959d0a8220c9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:39.852854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 961/1927] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:39 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:39.858618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5b093-6fc9-450c-ad58-3f2445f5cee8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:39.869658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5b093-6fc9-450c-ad58-3f2445f5cee8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:39.929608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fadf94-3cf0-432c-aad3-13ed948acb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:39.930349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fadf94-3cf0-432c-aad3-13ed948acb94 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:39.930635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fadf94-3cf0-432c-aad3-13ed948acb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:39.930917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fadf94-3cf0-432c-aad3-13ed948acb94 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:39.931626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fadf94-3cf0-432c-aad3-13ed948acb94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:39.933545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fadf94-3cf0-432c-aad3-13ed948acb94 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:39.956973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 967/1928] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:39 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:39.960491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fb16f-f6fe-4765-b322-959d0a8220c9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:39.970733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fb16f-f6fe-4765-b322-959d0a8220c9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:39.976247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765006f-c234-4a71-9fd7-c467512c6263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:39.976744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765006f-c234-4a71-9fd7-c467512c6263 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:39.977023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765006f-c234-4a71-9fd7-c467512c6263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:39.977306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765006f-c234-4a71-9fd7-c467512c6263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:39.978087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765006f-c234-4a71-9fd7-c467512c6263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:39.979683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765006f-c234-4a71-9fd7-c467512c6263 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:39.992940 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 962/1929] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:39.997027 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fadf94-3cf0-432c-aad3-13ed948acb94 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:40.010442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fadf94-3cf0-432c-aad3-13ed948acb94 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:40.016782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1b77a-9343-45b6-82b3-ae5b490bd588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:40.017536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1b77a-9343-45b6-82b3-ae5b490bd588 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:40.017870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1b77a-9343-45b6-82b3-ae5b490bd588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:40.018192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1b77a-9343-45b6-82b3-ae5b490bd588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:40.019011 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1b77a-9343-45b6-82b3-ae5b490bd588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:40.020666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1b77a-9343-45b6-82b3-ae5b490bd588 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:40.034492 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 968/1930] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:40.039312 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765006f-c234-4a71-9fd7-c467512c6263 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:40.049898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765006f-c234-4a71-9fd7-c467512c6263 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:40.055992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41791cd0-f0da-4a1c-b503-d47a8b758588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:40.057049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41791cd0-f0da-4a1c-b503-d47a8b758588 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:40.057339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41791cd0-f0da-4a1c-b503-d47a8b758588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:40.057616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41791cd0-f0da-4a1c-b503-d47a8b758588 admin admin] PATH_INFO: `/v3/projects/86bdfe3cff6047bd98ccfb9bc60bd435/users/beb0eda9888448488d4ac40fab43472d/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:40.078751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41791cd0-f0da-4a1c-b503-d47a8b758588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bdfe3cff6047bd98ccfb9bc60bd435, user_id=beb0eda9888448488d4ac40fab43472d, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:40.081551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41791cd0-f0da-4a1c-b503-d47a8b758588 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:40.096524 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 963/1931] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:40 2022] PUT /identity/v3/projects/86bdfe3cff6047bd98ccfb9bc60bd435/users/beb0eda9888448488d4ac40fab43472d/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:40.110202 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a421ea1-782e-48bb-9e71-20a983c825bb None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:40.110571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a421ea1-782e-48bb-9e71-20a983c825bb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:40.110915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a421ea1-782e-48bb-9e71-20a983c825bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:40.142950 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:40.148404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0a421ea1-782e-48bb-9e71-20a983c825bb None None] MFA Rules not processed for user `beb0eda9888448488d4ac40fab43472d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:40.150789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0a421ea1-782e-48bb-9e71-20a983c825bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:40.224686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 969/1932] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:40.229864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41791cd0-f0da-4a1c-b503-d47a8b758588 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:40.305566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41791cd0-f0da-4a1c-b503-d47a8b758588 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:40.310411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4f855-2fcf-46fc-85fc-7cb4289df74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:40.311075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4f855-2fcf-46fc-85fc-7cb4289df74e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:40.311362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4f855-2fcf-46fc-85fc-7cb4289df74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:40.311642 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4f855-2fcf-46fc-85fc-7cb4289df74e admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:40.312338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4f855-2fcf-46fc-85fc-7cb4289df74e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:40.314271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4f855-2fcf-46fc-85fc-7cb4289df74e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:40.339018 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 964/1933] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:40 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:40.342864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a421ea1-782e-48bb-9e71-20a983c825bb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:40.355679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a421ea1-782e-48bb-9e71-20a983c825bb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:40.413462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02979b19-a4db-4710-be4a-78a315993afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:40.414304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02979b19-a4db-4710-be4a-78a315993afa admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:40.414631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02979b19-a4db-4710-be4a-78a315993afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:40.415445 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02979b19-a4db-4710-be4a-78a315993afa admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:40.416233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02979b19-a4db-4710-be4a-78a315993afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:40.418379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02979b19-a4db-4710-be4a-78a315993afa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:40.438863 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 970/1934] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:40 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:40.442861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4f855-2fcf-46fc-85fc-7cb4289df74e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:40.453297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4f855-2fcf-46fc-85fc-7cb4289df74e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:40.457896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062416e-5158-45fb-a266-5a3137b58cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:40.458623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062416e-5158-45fb-a266-5a3137b58cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:40.458931 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062416e-5158-45fb-a266-5a3137b58cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:40.459769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062416e-5158-45fb-a266-5a3137b58cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:40.460504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062416e-5158-45fb-a266-5a3137b58cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:40.462145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062416e-5158-45fb-a266-5a3137b58cd9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:40.472645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 965/1935] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:40.477607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02979b19-a4db-4710-be4a-78a315993afa admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:40.488151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02979b19-a4db-4710-be4a-78a315993afa admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:40.493125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3b15e-a277-490d-a34f-b5f2a5d5fdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:40.494055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3b15e-a277-490d-a34f-b5f2a5d5fdc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:40.494348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3b15e-a277-490d-a34f-b5f2a5d5fdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:40.494632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3b15e-a277-490d-a34f-b5f2a5d5fdc8 admin admin] PATH_INFO: `/v3/projects/888dfccbaf6d4df8bb11e0d2c49f159b/users/ed2e580d324b4699891b3ae5e7c2b548/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:40.509359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3b15e-a277-490d-a34f-b5f2a5d5fdc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=888dfccbaf6d4df8bb11e0d2c49f159b, user_id=ed2e580d324b4699891b3ae5e7c2b548, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:40.512049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3b15e-a277-490d-a34f-b5f2a5d5fdc8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:40.524848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 971/1936] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:40 2022] PUT /identity/v3/projects/888dfccbaf6d4df8bb11e0d2c49f159b/users/ed2e580d324b4699891b3ae5e7c2b548/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:40.529000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062416e-5158-45fb-a266-5a3137b58cd9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:40.540249 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062416e-5158-45fb-a266-5a3137b58cd9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:40.588187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:40.599352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad5808-f6d5-4c60-b52f-b8b3daaac137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:40.600943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad5808-f6d5-4c60-b52f-b8b3daaac137 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:40.601313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad5808-f6d5-4c60-b52f-b8b3daaac137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:40.601590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad5808-f6d5-4c60-b52f-b8b3daaac137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:40.602360 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad5808-f6d5-4c60-b52f-b8b3daaac137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:40.603972 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad5808-f6d5-4c60-b52f-b8b3daaac137 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:40.613781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 966/1937] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:40.618381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3b15e-a277-490d-a34f-b5f2a5d5fdc8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:40.628655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3b15e-a277-490d-a34f-b5f2a5d5fdc8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:40.633693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e8090-71f1-4f7f-9ede-82a0d5d51f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:40.634612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e8090-71f1-4f7f-9ede-82a0d5d51f7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:40.634922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e8090-71f1-4f7f-9ede-82a0d5d51f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:40.635188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e8090-71f1-4f7f-9ede-82a0d5d51f7d admin admin] PATH_INFO: `/v3/projects/888dfccbaf6d4df8bb11e0d2c49f159b/users/ed2e580d324b4699891b3ae5e7c2b548/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:40.637210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e8090-71f1-4f7f-9ede-82a0d5d51f7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=888dfccbaf6d4df8bb11e0d2c49f159b, user_id=ed2e580d324b4699891b3ae5e7c2b548, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:40.639894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e8090-71f1-4f7f-9ede-82a0d5d51f7d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:40.651206 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 972/1938] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:40 2022] PUT /identity/v3/projects/888dfccbaf6d4df8bb11e0d2c49f159b/users/ed2e580d324b4699891b3ae5e7c2b548/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:40.661318 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57581034-ff3f-4c11-b7b7-5e9b864a52f8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:40.661668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57581034-ff3f-4c11-b7b7-5e9b864a52f8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:40.662030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57581034-ff3f-4c11-b7b7-5e9b864a52f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:40.690443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-57581034-ff3f-4c11-b7b7-5e9b864a52f8 None None] MFA Rules not processed for user `ed2e580d324b4699891b3ae5e7c2b548`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:40.691639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-57581034-ff3f-4c11-b7b7-5e9b864a52f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:40.762468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 967/1939] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:40.776839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e8090-71f1-4f7f-9ede-82a0d5d51f7d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:40.778588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-859e8090-71f1-4f7f-9ede-82a0d5d51f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:40.842155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e8090-71f1-4f7f-9ede-82a0d5d51f7d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:40.847091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c545f7-ac8b-4dda-ad74-966ed96dc902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:40.847091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c545f7-ac8b-4dda-ad74-966ed96dc902 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:40.847425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c545f7-ac8b-4dda-ad74-966ed96dc902 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:40.849713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c545f7-ac8b-4dda-ad74-966ed96dc902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:40.869020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c545f7-ac8b-4dda-ad74-966ed96dc902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:40.870693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c545f7-ac8b-4dda-ad74-966ed96dc902 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:40.875183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:40.886152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 973/1940] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:40.981207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57581034-ff3f-4c11-b7b7-5e9b864a52f8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:40.982737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-57581034-ff3f-4c11-b7b7-5e9b864a52f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:40.996090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57581034-ff3f-4c11-b7b7-5e9b864a52f8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:41.000072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc70c3-2636-4a73-97b6-8eba0da03f9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:41.000737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc70c3-2636-4a73-97b6-8eba0da03f9e service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:41.001020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc70c3-2636-4a73-97b6-8eba0da03f9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:41.001298 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc70c3-2636-4a73-97b6-8eba0da03f9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:41.106217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc70c3-2636-4a73-97b6-8eba0da03f9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:41.108676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc70c3-2636-4a73-97b6-8eba0da03f9e service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:41.120451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 968/1941] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:41.429391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c545f7-ac8b-4dda-ad74-966ed96dc902 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:41.494440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c545f7-ac8b-4dda-ad74-966ed96dc902 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:41.498941 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:41.499811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:41.500099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:41.500396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] PATH_INFO: `/v3/users/beb0eda9888448488d4ac40fab43472d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:41.501567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beb0eda9888448488d4ac40fab43472d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:41.503513 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:41.550676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beb0eda9888448488d4ac40fab43472d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:41.562592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beb0eda9888448488d4ac40fab43472d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:41.569967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beb0eda9888448488d4ac40fab43472d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:41.570654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beb0eda9888448488d4ac40fab43472d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:41.571260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beb0eda9888448488d4ac40fab43472d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:41.585828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 974/1942] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:41 2022] DELETE /identity/v3/users/beb0eda9888448488d4ac40fab43472d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:41.590397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505b046-15d6-46ab-b0e5-e665beec3aeb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:41.590692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505b046-15d6-46ab-b0e5-e665beec3aeb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:41.590986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505b046-15d6-46ab-b0e5-e665beec3aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:41.608357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e505b046-15d6-46ab-b0e5-e665beec3aeb None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:41.609481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e505b046-15d6-46ab-b0e5-e665beec3aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:41.680925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 969/1943] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:41.690364 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:41.691747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:41.761872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0914640-4f54-4812-8514-08708d8fb0b4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:41.765487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e40f-e501-43db-a0d6-cd8605304089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:41.766179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e40f-e501-43db-a0d6-cd8605304089 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:41.766463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e40f-e501-43db-a0d6-cd8605304089 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:41.766741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e40f-e501-43db-a0d6-cd8605304089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:41.776592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87e40f-e501-43db-a0d6-cd8605304089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:41.778083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87e40f-e501-43db-a0d6-cd8605304089 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:41.789107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 975/1944] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:42.065895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505b046-15d6-46ab-b0e5-e665beec3aeb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:42.068130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e505b046-15d6-46ab-b0e5-e665beec3aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:42.088256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505b046-15d6-46ab-b0e5-e665beec3aeb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:42.093329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:42.094434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:42.094721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:42.095004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] PATH_INFO: `/v3/projects/86bdfe3cff6047bd98ccfb9bc60bd435` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:42.096235 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=86bdfe3cff6047bd98ccfb9bc60bd435)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:42.098184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:42.106972 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:42.163937 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86bdfe3cff6047bd98ccfb9bc60bd435', 'request_id': 'req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:42.170179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86bdfe3cff6047bd98ccfb9bc60bd435', 'request_id': 'req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:42.182917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86bdfe3cff6047bd98ccfb9bc60bd435', 'request_id': 'req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:42.199469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] The token cache is being invalidate because project 86bdfe3cff6047bd98ccfb9bc60bd435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:42.199895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:42.200677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:42.202597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 970/1945] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:42 2022] DELETE /identity/v3/projects/86bdfe3cff6047bd98ccfb9bc60bd435 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:42.206310 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e40f-e501-43db-a0d6-cd8605304089 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:42.218833 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e40f-e501-43db-a0d6-cd8605304089 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:42.245527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:42.299778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80e440-8bd6-4769-854c-599ddb3bdeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:42.300673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80e440-8bd6-4769-854c-599ddb3bdeb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:42.300960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80e440-8bd6-4769-854c-599ddb3bdeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:42.301245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80e440-8bd6-4769-854c-599ddb3bdeb4 admin admin] PATH_INFO: `/v3/users/ed2e580d324b4699891b3ae5e7c2b548` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:42.302545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80e440-8bd6-4769-854c-599ddb3bdeb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed2e580d324b4699891b3ae5e7c2b548)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:42.305220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80e440-8bd6-4769-854c-599ddb3bdeb4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:42.355438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd80e440-8bd6-4769-854c-599ddb3bdeb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed2e580d324b4699891b3ae5e7c2b548'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:42.370849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd80e440-8bd6-4769-854c-599ddb3bdeb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed2e580d324b4699891b3ae5e7c2b548'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:42.377997 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd80e440-8bd6-4769-854c-599ddb3bdeb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed2e580d324b4699891b3ae5e7c2b548'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:42.378737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd80e440-8bd6-4769-854c-599ddb3bdeb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed2e580d324b4699891b3ae5e7c2b548'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:42.379405 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd80e440-8bd6-4769-854c-599ddb3bdeb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed2e580d324b4699891b3ae5e7c2b548'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:42.395118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 976/1946] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:42 2022] DELETE /identity/v3/users/ed2e580d324b4699891b3ae5e7c2b548 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:43.073145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:43.075395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:43.097492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af8b3-48d9-4b9e-bc28-ca0f23d04a8d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:43.175205 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:43.176190 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:43.176481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:43.176766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] PATH_INFO: `/v3/projects/888dfccbaf6d4df8bb11e0d2c49f159b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:43.178147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=888dfccbaf6d4df8bb11e0d2c49f159b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:43.180170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:43.248115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '888dfccbaf6d4df8bb11e0d2c49f159b', 'request_id': 'req-56efbf22-5a43-4203-93d9-4c63152428d1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:43.275614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '888dfccbaf6d4df8bb11e0d2c49f159b', 'request_id': 'req-56efbf22-5a43-4203-93d9-4c63152428d1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:43.289011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '888dfccbaf6d4df8bb11e0d2c49f159b', 'request_id': 'req-56efbf22-5a43-4203-93d9-4c63152428d1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:43.314450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] The token cache is being invalidate because project 888dfccbaf6d4df8bb11e0d2c49f159b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:43.314660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:43.316743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56efbf22-5a43-4203-93d9-4c63152428d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:43.321142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 971/1947] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:43 2022] DELETE /identity/v3/projects/888dfccbaf6d4df8bb11e0d2c49f159b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:43.328440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ea648-d66c-4e75-a82b-73ac374737ed None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:43.330903 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ea648-d66c-4e75-a82b-73ac374737ed None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:43.331419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ea648-d66c-4e75-a82b-73ac374737ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:43.358627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8d0ea648-d66c-4e75-a82b-73ac374737ed None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:43.360263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8d0ea648-d66c-4e75-a82b-73ac374737ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:43.439092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 977/1948] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:43.447894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41fe37-ffdd-44a8-907a-f371399f35c6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:43.448187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41fe37-ffdd-44a8-907a-f371399f35c6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:43.448470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41fe37-ffdd-44a8-907a-f371399f35c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:43.468400 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ba41fe37-ffdd-44a8-907a-f371399f35c6 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:43.469623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ba41fe37-ffdd-44a8-907a-f371399f35c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:43.478076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 972/1949] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:43.482224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ea648-d66c-4e75-a82b-73ac374737ed None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:43.492459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ea648-d66c-4e75-a82b-73ac374737ed None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:43.497072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d27562-9e61-4eb2-80b0-6774190918d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:43.497739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d27562-9e61-4eb2-80b0-6774190918d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:43.498055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d27562-9e61-4eb2-80b0-6774190918d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:43.498340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d27562-9e61-4eb2-80b0-6774190918d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:43.499077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d27562-9e61-4eb2-80b0-6774190918d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:43.499388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d27562-9e61-4eb2-80b0-6774190918d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:43.500990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d27562-9e61-4eb2-80b0-6774190918d3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:43.513616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 978/1950] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:43.518236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41fe37-ffdd-44a8-907a-f371399f35c6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:43.529541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41fe37-ffdd-44a8-907a-f371399f35c6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:43.534637 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ace53-d81d-43d8-a066-4275a86566d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:43.535345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ace53-d81d-43d8-a066-4275a86566d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:43.535633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ace53-d81d-43d8-a066-4275a86566d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:43.535916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ace53-d81d-43d8-a066-4275a86566d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:43.536619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ace53-d81d-43d8-a066-4275a86566d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:43.538893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ace53-d81d-43d8-a066-4275a86566d1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:43.556787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:43.573920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 973/1951] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:43 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:43.577792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d27562-9e61-4eb2-80b0-6774190918d3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:43.588823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d27562-9e61-4eb2-80b0-6774190918d3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:43.622017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:43.674938 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2951-b548-42b4-a2ab-cf964755e15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:43.675899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2951-b548-42b4-a2ab-cf964755e15f admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:43.676440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2951-b548-42b4-a2ab-cf964755e15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:43.676959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2951-b548-42b4-a2ab-cf964755e15f admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:43.678684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b2951-b548-42b4-a2ab-cf964755e15f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:43.682351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b2951-b548-42b4-a2ab-cf964755e15f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:43.713223 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 979/1952] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:43 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:43.714645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ace53-d81d-43d8-a066-4275a86566d1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:43.748234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ace53-d81d-43d8-a066-4275a86566d1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:43.760778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f2034-06e0-4c31-8b99-d47c36c298c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:43.761615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f2034-06e0-4c31-8b99-d47c36c298c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:43.761902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f2034-06e0-4c31-8b99-d47c36c298c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:43.762398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f2034-06e0-4c31-8b99-d47c36c298c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:43.765340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f2034-06e0-4c31-8b99-d47c36c298c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:43.767560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f2034-06e0-4c31-8b99-d47c36c298c3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:43.782218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 974/1953] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:43.786334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2951-b548-42b4-a2ab-cf964755e15f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:43.798234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2951-b548-42b4-a2ab-cf964755e15f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:43.803634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c998f-fa43-414d-bb24-c60d8bf127e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:43.804433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c998f-fa43-414d-bb24-c60d8bf127e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:43.804999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c998f-fa43-414d-bb24-c60d8bf127e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:43.805311 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c998f-fa43-414d-bb24-c60d8bf127e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:43.806087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c998f-fa43-414d-bb24-c60d8bf127e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:43.807779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c998f-fa43-414d-bb24-c60d8bf127e5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:43.819499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 980/1954] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:43.823748 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f2034-06e0-4c31-8b99-d47c36c298c3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:43.833997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f2034-06e0-4c31-8b99-d47c36c298c3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:43.839626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a5703-4ff9-4cce-9edc-4aab63abca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:43.840547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a5703-4ff9-4cce-9edc-4aab63abca1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:43.840826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a5703-4ff9-4cce-9edc-4aab63abca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:43.841096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a5703-4ff9-4cce-9edc-4aab63abca1c admin admin] PATH_INFO: `/v3/projects/f178cec329fb43198c0ab83e3b4714cc/users/39e2d32aab85469da14c929ce1a7abea/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:43.860170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a5703-4ff9-4cce-9edc-4aab63abca1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f178cec329fb43198c0ab83e3b4714cc, user_id=39e2d32aab85469da14c929ce1a7abea, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:43.863000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a5703-4ff9-4cce-9edc-4aab63abca1c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:43.883262 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 975/1955] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:43 2022] PUT /identity/v3/projects/f178cec329fb43198c0ab83e3b4714cc/users/39e2d32aab85469da14c929ce1a7abea/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:43.895315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83b2eb-3525-4b23-8658-7dc09503973b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:43.895894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83b2eb-3525-4b23-8658-7dc09503973b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:43.896362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83b2eb-3525-4b23-8658-7dc09503973b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:43.925462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bf83b2eb-3525-4b23-8658-7dc09503973b None None] MFA Rules not processed for user `39e2d32aab85469da14c929ce1a7abea`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:43.926915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bf83b2eb-3525-4b23-8658-7dc09503973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:43.993836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 981/1956] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:44.005450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a5703-4ff9-4cce-9edc-4aab63abca1c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:44.007761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c97a5703-4ff9-4cce-9edc-4aab63abca1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:44.069375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a5703-4ff9-4cce-9edc-4aab63abca1c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:44.072943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621bb4a-0117-46f0-9fe1-6a4b8ef6cac0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:44.073594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621bb4a-0117-46f0-9fe1-6a4b8ef6cac0 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:44.074185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621bb4a-0117-46f0-9fe1-6a4b8ef6cac0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:44.074185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621bb4a-0117-46f0-9fe1-6a4b8ef6cac0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:44.083121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621bb4a-0117-46f0-9fe1-6a4b8ef6cac0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:44.085257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621bb4a-0117-46f0-9fe1-6a4b8ef6cac0 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:44.094464 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 976/1957] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:46.454696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9fed9-dd9f-4028-b117-c384f231c831 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:46.455238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9fed9-dd9f-4028-b117-c384f231c831 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:46.455358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9fed9-dd9f-4028-b117-c384f231c831 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:46.456866 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 982/1958] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 06:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:31:46.490403 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8d4d7-ec85-4526-8ce7-c90bf95733a0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:46.490895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8d4d7-ec85-4526-8ce7-c90bf95733a0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:46.491566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8d4d7-ec85-4526-8ce7-c90bf95733a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:46.514867 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8ba8d4d7-ec85-4526-8ce7-c90bf95733a0 None None] MFA Rules not processed for user `1da2d85eb18049f196aeefc51d3f1e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:46.516675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8ba8d4d7-ec85-4526-8ce7-c90bf95733a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:46.534906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9fed9-dd9f-4028-b117-c384f231c831 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:46.538890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-90a9fed9-dd9f-4028-b117-c384f231c831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:46.596835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 977/1959] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 06:31:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:46.608875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8d4d7-ec85-4526-8ce7-c90bf95733a0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:46.610343 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8ba8d4d7-ec85-4526-8ce7-c90bf95733a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:46.621402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9fed9-dd9f-4028-b117-c384f231c831 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:46.623450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8d4d7-ec85-4526-8ce7-c90bf95733a0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:46.626005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d548c24-7211-47dc-9d0f-ddcf924481e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:46.626895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d548c24-7211-47dc-9d0f-ddcf924481e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:46.627381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d548c24-7211-47dc-9d0f-ddcf924481e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:46.627744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d548c24-7211-47dc-9d0f-ddcf924481e6 admin admin] PATH_INFO: `/v3/users/dfeb642f25484665b7c477cde153162d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:46.628981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d548c24-7211-47dc-9d0f-ddcf924481e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfeb642f25484665b7c477cde153162d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:46.629821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d73c0-9ea1-4662-a406-192b9c8ec62f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da2d85eb18049f196aeefc51d3f1e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:46.630655 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d73c0-9ea1-4662-a406-192b9c8ec62f service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:46.630966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d73c0-9ea1-4662-a406-192b9c8ec62f service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:46.631168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d548c24-7211-47dc-9d0f-ddcf924481e6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:46.631264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d73c0-9ea1-4662-a406-192b9c8ec62f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:46.641059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d73c0-9ea1-4662-a406-192b9c8ec62f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:46.642714 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d73c0-9ea1-4662-a406-192b9c8ec62f service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:46.657454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 978/1960] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:46.680698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:46.692919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d73c0-9ea1-4662-a406-192b9c8ec62f service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:46.695930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2d548c24-7211-47dc-9d0f-ddcf924481e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfeb642f25484665b7c477cde153162d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:46.707122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d73c0-9ea1-4662-a406-192b9c8ec62f service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:46.710848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2d548c24-7211-47dc-9d0f-ddcf924481e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfeb642f25484665b7c477cde153162d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:46.714570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:46.715719 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:46.716397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:46.716918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] PATH_INFO: `/v3/users/39e2d32aab85469da14c929ce1a7abea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:46.719079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e2d32aab85469da14c929ce1a7abea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:46.719195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2d548c24-7211-47dc-9d0f-ddcf924481e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfeb642f25484665b7c477cde153162d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:46.719844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2d548c24-7211-47dc-9d0f-ddcf924481e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfeb642f25484665b7c477cde153162d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:46.721076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2d548c24-7211-47dc-9d0f-ddcf924481e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfeb642f25484665b7c477cde153162d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:46.722085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:46.745676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 983/1961] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:46 2022] DELETE /identity/v3/users/dfeb642f25484665b7c477cde153162d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:46.794944 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:46.803579 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e2d32aab85469da14c929ce1a7abea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:46.805855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e2d32aab85469da14c929ce1a7abea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:46.817529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e2d32aab85469da14c929ce1a7abea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:46.838496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e2d32aab85469da14c929ce1a7abea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:46.839353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e2d32aab85469da14c929ce1a7abea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:46.857039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 979/1962] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:46 2022] DELETE /identity/v3/users/39e2d32aab85469da14c929ce1a7abea => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:46.861659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4f1d-7281-492a-853f-da9b943cc235 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:46.861980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4f1d-7281-492a-853f-da9b943cc235 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:46.862258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4f1d-7281-492a-853f-da9b943cc235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:46.887805 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-efda4f1d-7281-492a-853f-da9b943cc235 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:46.890055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-efda4f1d-7281-492a-853f-da9b943cc235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:46.949388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:46.951324 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:46.997766 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 984/1963] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:47.001177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda4f1d-7281-492a-853f-da9b943cc235 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.002756 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-efda4f1d-7281-492a-853f-da9b943cc235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:47.063191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0948a-9224-4172-9aab-6f5ede62bed0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.070313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.071197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.071482 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.071761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] PATH_INFO: `/v3/projects/c743e22f3fdb4a61ad1b8035ac06d15b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.073452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c743e22f3fdb4a61ad1b8035ac06d15b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.075429 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.097589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda4f1d-7281-492a-853f-da9b943cc235 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.104190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d006e5-8414-4435-bd7b-94a0b573c59b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.104859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d006e5-8414-4435-bd7b-94a0b573c59b service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.105151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d006e5-8414-4435-bd7b-94a0b573c59b service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.105432 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d006e5-8414-4435-bd7b-94a0b573c59b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.123302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d006e5-8414-4435-bd7b-94a0b573c59b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.124863 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d006e5-8414-4435-bd7b-94a0b573c59b service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.138574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c743e22f3fdb4a61ad1b8035ac06d15b', 'request_id': 'req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:47.144446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c743e22f3fdb4a61ad1b8035ac06d15b', 'request_id': 'req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:47.145404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 985/1964] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:31:47.152180 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c743e22f3fdb4a61ad1b8035ac06d15b', 'request_id': 'req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:47.169605 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] The token cache is being invalidate because project c743e22f3fdb4a61ad1b8035ac06d15b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:47.170082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:47.170797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6cf237c7-4e1d-4f52-ae1f-a387308fae9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:47.173733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 980/1965] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:46 2022] DELETE /identity/v3/projects/c743e22f3fdb4a61ad1b8035ac06d15b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:47.179323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17466d90-be92-4efa-936d-d4284e2f35a0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.179910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17466d90-be92-4efa-936d-d4284e2f35a0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.180290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17466d90-be92-4efa-936d-d4284e2f35a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.200981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-17466d90-be92-4efa-936d-d4284e2f35a0 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:47.203633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-17466d90-be92-4efa-936d-d4284e2f35a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:47.266622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:47.281987 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 986/1966] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:47.289248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1474-7823-404e-85df-d6a0bf1a4fd2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.289860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1474-7823-404e-85df-d6a0bf1a4fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.290024 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1474-7823-404e-85df-d6a0bf1a4fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.321879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-445a1474-7823-404e-85df-d6a0bf1a4fd2 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:47.324219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-445a1474-7823-404e-85df-d6a0bf1a4fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:47.338498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 981/1967] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:47.339742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17466d90-be92-4efa-936d-d4284e2f35a0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.351507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17466d90-be92-4efa-936d-d4284e2f35a0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.357806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca49b06-1ee6-463b-b435-a5fb3d339948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.358738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca49b06-1ee6-463b-b435-a5fb3d339948 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.359045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca49b06-1ee6-463b-b435-a5fb3d339948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.359407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca49b06-1ee6-463b-b435-a5fb3d339948 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.360299 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca49b06-1ee6-463b-b435-a5fb3d339948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:47.360887 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca49b06-1ee6-463b-b435-a5fb3d339948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.362876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca49b06-1ee6-463b-b435-a5fb3d339948 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.382215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 987/1968] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:47.386791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a1474-7823-404e-85df-d6a0bf1a4fd2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.404570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a1474-7823-404e-85df-d6a0bf1a4fd2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.409302 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af71980-c8dc-41e6-851a-3a889ec3eb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.410152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af71980-c8dc-41e6-851a-3a889ec3eb08 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.410454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af71980-c8dc-41e6-851a-3a889ec3eb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.410727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af71980-c8dc-41e6-851a-3a889ec3eb08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.411421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af71980-c8dc-41e6-851a-3a889ec3eb08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.414280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af71980-c8dc-41e6-851a-3a889ec3eb08 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.445996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 982/1969] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:47 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:47.448062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca49b06-1ee6-463b-b435-a5fb3d339948 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.466181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca49b06-1ee6-463b-b435-a5fb3d339948 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.530462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea8b51-3d31-45c5-9814-a63a61194c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.531157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea8b51-3d31-45c5-9814-a63a61194c81 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.531447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea8b51-3d31-45c5-9814-a63a61194c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.531751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea8b51-3d31-45c5-9814-a63a61194c81 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.532468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea8b51-3d31-45c5-9814-a63a61194c81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.534434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea8b51-3d31-45c5-9814-a63a61194c81 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.555678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 988/1970] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:47 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:47.559947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af71980-c8dc-41e6-851a-3a889ec3eb08 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.570826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af71980-c8dc-41e6-851a-3a889ec3eb08 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.575776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af457-d1d8-4221-b122-ce08775ef369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.576739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af457-d1d8-4221-b122-ce08775ef369 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.577171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af457-d1d8-4221-b122-ce08775ef369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.577516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af457-d1d8-4221-b122-ce08775ef369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.578505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9af457-d1d8-4221-b122-ce08775ef369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.581973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9af457-d1d8-4221-b122-ce08775ef369 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.598025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea8b51-3d31-45c5-9814-a63a61194c81 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.599027 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 983/1971] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:47.602962 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af457-d1d8-4221-b122-ce08775ef369 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.605614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ba9af457-d1d8-4221-b122-ce08775ef369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:47.609694 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea8b51-3d31-45c5-9814-a63a61194c81 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.615227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa931dd2-3b43-4184-9ed3-480762643426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.615915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa931dd2-3b43-4184-9ed3-480762643426 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.616197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa931dd2-3b43-4184-9ed3-480762643426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.616474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa931dd2-3b43-4184-9ed3-480762643426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.618096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa931dd2-3b43-4184-9ed3-480762643426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.619902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af457-d1d8-4221-b122-ce08775ef369 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.620442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa931dd2-3b43-4184-9ed3-480762643426 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.625187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.626196 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.626484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.626768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] PATH_INFO: `/v3/projects/f178cec329fb43198c0ab83e3b4714cc` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.628184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f178cec329fb43198c0ab83e3b4714cc)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.630132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.632787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 989/1972] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:47.633857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:47.637157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa931dd2-3b43-4184-9ed3-480762643426 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.648483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa931dd2-3b43-4184-9ed3-480762643426 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.653622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131f184-8289-45ab-89b0-6afeb51a9ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.654712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131f184-8289-45ab-89b0-6afeb51a9ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.655001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131f184-8289-45ab-89b0-6afeb51a9ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.655292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131f184-8289-45ab-89b0-6afeb51a9ee3 admin admin] PATH_INFO: `/v3/projects/6f4dbe3626554145871b7c889317fbce/users/ebbee9b2f89f4f56959cfc5c0a1925c5/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.671183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131f184-8289-45ab-89b0-6afeb51a9ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4dbe3626554145871b7c889317fbce, user_id=ebbee9b2f89f4f56959cfc5c0a1925c5, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.674190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131f184-8289-45ab-89b0-6afeb51a9ee3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.682849 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f178cec329fb43198c0ab83e3b4714cc', 'request_id': 'req-399f70f7-0230-420a-bf55-110efb0c7cb8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:47.696879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f178cec329fb43198c0ab83e3b4714cc', 'request_id': 'req-399f70f7-0230-420a-bf55-110efb0c7cb8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:47.697055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 990/1973] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:47 2022] PUT /identity/v3/projects/6f4dbe3626554145871b7c889317fbce/users/ebbee9b2f89f4f56959cfc5c0a1925c5/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:47.697154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f178cec329fb43198c0ab83e3b4714cc', 'request_id': 'req-399f70f7-0230-420a-bf55-110efb0c7cb8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:47.699750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19877fcf-7937-4ecb-bfd9-e61ea44bdc88 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.700189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19877fcf-7937-4ecb-bfd9-e61ea44bdc88 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.700504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19877fcf-7937-4ecb-bfd9-e61ea44bdc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.710160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] The token cache is being invalidate because project f178cec329fb43198c0ab83e3b4714cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:47.710451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:47.711103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-399f70f7-0230-420a-bf55-110efb0c7cb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:47.713036 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 984/1974] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:47 2022] DELETE /identity/v3/projects/f178cec329fb43198c0ab83e3b4714cc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:47.717614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee81d5-5e3d-4d0a-8241-a3f9fe98bb93 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.718204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee81d5-5e3d-4d0a-8241-a3f9fe98bb93 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.718587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee81d5-5e3d-4d0a-8241-a3f9fe98bb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.730278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-19877fcf-7937-4ecb-bfd9-e61ea44bdc88 None None] MFA Rules not processed for user `ebbee9b2f89f4f56959cfc5c0a1925c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:47.731506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-19877fcf-7937-4ecb-bfd9-e61ea44bdc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:47.738952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7aee81d5-5e3d-4d0a-8241-a3f9fe98bb93 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:47.740297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7aee81d5-5e3d-4d0a-8241-a3f9fe98bb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:47.806573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 991/1975] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:47.806761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 985/1976] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:47.807875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19877fcf-7937-4ecb-bfd9-e61ea44bdc88 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.809457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-19877fcf-7937-4ecb-bfd9-e61ea44bdc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:47.813583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816188-e480-4ebe-b852-f4ed43055ce1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.814093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816188-e480-4ebe-b852-f4ed43055ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.814384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816188-e480-4ebe-b852-f4ed43055ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.825070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19877fcf-7937-4ecb-bfd9-e61ea44bdc88 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.829916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd3a5f-4433-4cfb-a631-d0673c06ff73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.830636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd3a5f-4433-4cfb-a631-d0673c06ff73 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.830919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd3a5f-4433-4cfb-a631-d0673c06ff73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.831195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd3a5f-4433-4cfb-a631-d0673c06ff73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.831885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd3a5f-4433-4cfb-a631-d0673c06ff73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.833623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-08816188-e480-4ebe-b852-f4ed43055ce1 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:47.834500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd3a5f-4433-4cfb-a631-d0673c06ff73 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.835523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-08816188-e480-4ebe-b852-f4ed43055ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:47.843556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 986/1977] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:47.848422 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08816188-e480-4ebe-b852-f4ed43055ce1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.860273 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08816188-e480-4ebe-b852-f4ed43055ce1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.862781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 992/1978] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:47 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:47.867080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fe550-cf13-46f4-86b9-0778b1051023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.867365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd3a5f-4433-4cfb-a631-d0673c06ff73 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.867895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fe550-cf13-46f4-86b9-0778b1051023 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.868183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fe550-cf13-46f4-86b9-0778b1051023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.868461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fe550-cf13-46f4-86b9-0778b1051023 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.869189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fe550-cf13-46f4-86b9-0778b1051023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:47.869495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fe550-cf13-46f4-86b9-0778b1051023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.871160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fe550-cf13-46f4-86b9-0778b1051023 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.878318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:47.886822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 987/1979] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:47.891205 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fe550-cf13-46f4-86b9-0778b1051023 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.902203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fe550-cf13-46f4-86b9-0778b1051023 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.937854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd3a5f-4433-4cfb-a631-d0673c06ff73 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.942568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771074e-4a43-4c3c-bbd7-6328e68e5256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.943881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771074e-4a43-4c3c-bbd7-6328e68e5256 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.944173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771074e-4a43-4c3c-bbd7-6328e68e5256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.944447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771074e-4a43-4c3c-bbd7-6328e68e5256 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.945150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771074e-4a43-4c3c-bbd7-6328e68e5256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.947094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771074e-4a43-4c3c-bbd7-6328e68e5256 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.964176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16483b99-ae18-45c9-b4e6-4cd5e5082bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.964877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16483b99-ae18-45c9-b4e6-4cd5e5082bce admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.965286 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16483b99-ae18-45c9-b4e6-4cd5e5082bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.965572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16483b99-ae18-45c9-b4e6-4cd5e5082bce admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.966347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16483b99-ae18-45c9-b4e6-4cd5e5082bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.969305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 993/1980] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:47 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:31:47.969399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16483b99-ae18-45c9-b4e6-4cd5e5082bce admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.970376 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771074e-4a43-4c3c-bbd7-6328e68e5256 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:47.980145 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771074e-4a43-4c3c-bbd7-6328e68e5256 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:47.984985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515897f7-bbc0-4904-9df6-85ef074e1b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:47.985649 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515897f7-bbc0-4904-9df6-85ef074e1b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:47.985931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515897f7-bbc0-4904-9df6-85ef074e1b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:47.986267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515897f7-bbc0-4904-9df6-85ef074e1b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:47.987145 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515897f7-bbc0-4904-9df6-85ef074e1b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:47.988930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515897f7-bbc0-4904-9df6-85ef074e1b57 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:47.989660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:48.003992 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 988/1981] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:47 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:31:48.004163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 994/1982] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:48.004830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515897f7-bbc0-4904-9df6-85ef074e1b57 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:48.007152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16483b99-ae18-45c9-b4e6-4cd5e5082bce admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:48.015400 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515897f7-bbc0-4904-9df6-85ef074e1b57 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:48.074509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16483b99-ae18-45c9-b4e6-4cd5e5082bce admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:48.079163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe59ca-8672-4b26-9e8f-ca68cb6165af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:48.079419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d80865-e104-48cd-8716-938397b242cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:48.080060 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe59ca-8672-4b26-9e8f-ca68cb6165af admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.080160 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d80865-e104-48cd-8716-938397b242cb admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.080345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe59ca-8672-4b26-9e8f-ca68cb6165af admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.080457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d80865-e104-48cd-8716-938397b242cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.080626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe59ca-8672-4b26-9e8f-ca68cb6165af admin admin] PATH_INFO: `/v3/projects/0d5e600cf1f6444c8c15d969aedec7b0/users/341ee814f7374844abae78f13c3b96cf/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.080720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d80865-e104-48cd-8716-938397b242cb admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.081333 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d80865-e104-48cd-8716-938397b242cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:48.083297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d80865-e104-48cd-8716-938397b242cb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:48.095582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe59ca-8672-4b26-9e8f-ca68cb6165af admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5e600cf1f6444c8c15d969aedec7b0, user_id=341ee814f7374844abae78f13c3b96cf, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:48.098280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe59ca-8672-4b26-9e8f-ca68cb6165af admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:48.106107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 995/1983] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:48 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:31:48.112033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 989/1984] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:48 2022] PUT /identity/v3/projects/0d5e600cf1f6444c8c15d969aedec7b0/users/341ee814f7374844abae78f13c3b96cf/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:48.112324 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d80865-e104-48cd-8716-938397b242cb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:48.117089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe59ca-8672-4b26-9e8f-ca68cb6165af admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:48.123558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d80865-e104-48cd-8716-938397b242cb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:48.181647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe59ca-8672-4b26-9e8f-ca68cb6165af admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:48.182636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb1bc5-8ae8-4212-9e1a-83919e88d46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:48.183378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb1bc5-8ae8-4212-9e1a-83919e88d46c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.183605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb1bc5-8ae8-4212-9e1a-83919e88d46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.183888 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb1bc5-8ae8-4212-9e1a-83919e88d46c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.184625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb1bc5-8ae8-4212-9e1a-83919e88d46c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:48.186244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb1bc5-8ae8-4212-9e1a-83919e88d46c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:48.186487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0eb81-74c4-461b-9698-a3ad57efeedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:48.187150 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0eb81-74c4-461b-9698-a3ad57efeedb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.187616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0eb81-74c4-461b-9698-a3ad57efeedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.187898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0eb81-74c4-461b-9698-a3ad57efeedb admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.189015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0eb81-74c4-461b-9698-a3ad57efeedb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:48.190646 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0eb81-74c4-461b-9698-a3ad57efeedb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:48.197020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 996/1985] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:48.200751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb1bc5-8ae8-4212-9e1a-83919e88d46c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:48.205682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 990/1986] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:48.206827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0eb81-74c4-461b-9698-a3ad57efeedb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:48.211388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb1bc5-8ae8-4212-9e1a-83919e88d46c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:48.216855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f2dec-963f-47b0-b328-6408d5495a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:48.217520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f2dec-963f-47b0-b328-6408d5495a6e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.217822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f2dec-963f-47b0-b328-6408d5495a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.218135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f2dec-963f-47b0-b328-6408d5495a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.218531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0eb81-74c4-461b-9698-a3ad57efeedb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:48.218988 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f2dec-963f-47b0-b328-6408d5495a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:48.220608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f2dec-963f-47b0-b328-6408d5495a6e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:48.223681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb71bc-1b43-4e00-ab0c-ee288d7b1bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:48.225084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb71bc-1b43-4e00-ab0c-ee288d7b1bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.225761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb71bc-1b43-4e00-ab0c-ee288d7b1bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.225761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb71bc-1b43-4e00-ab0c-ee288d7b1bb9 admin admin] PATH_INFO: `/v3/projects/0d5e600cf1f6444c8c15d969aedec7b0/users/341ee814f7374844abae78f13c3b96cf/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.226807 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:48.227756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb71bc-1b43-4e00-ab0c-ee288d7b1bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5e600cf1f6444c8c15d969aedec7b0, user_id=341ee814f7374844abae78f13c3b96cf, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:48.230838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb71bc-1b43-4e00-ab0c-ee288d7b1bb9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:48.233635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 997/1987] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:48.237663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f2dec-963f-47b0-b328-6408d5495a6e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:48.246930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 991/1988] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:48 2022] PUT /identity/v3/projects/0d5e600cf1f6444c8c15d969aedec7b0/users/341ee814f7374844abae78f13c3b96cf/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:48.247953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f2dec-963f-47b0-b328-6408d5495a6e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:48.257264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ed783-58ac-41d5-b278-9069f999887d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.257559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ed783-58ac-41d5-b278-9069f999887d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.257841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ed783-58ac-41d5-b278-9069f999887d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.284073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-533ed783-58ac-41d5-b278-9069f999887d None None] MFA Rules not processed for user `341ee814f7374844abae78f13c3b96cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:48.285333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-533ed783-58ac-41d5-b278-9069f999887d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:48.311586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a2b6f-557c-4fd6-8acf-9f4b8ae084c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:48.312494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a2b6f-557c-4fd6-8acf-9f4b8ae084c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.312778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a2b6f-557c-4fd6-8acf-9f4b8ae084c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.313060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a2b6f-557c-4fd6-8acf-9f4b8ae084c1 admin admin] PATH_INFO: `/v3/projects/3696b802980145649ec717ba8abcb02b/users/29a1ec39a0e24581b74e77df0a6478d9/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.328021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a2b6f-557c-4fd6-8acf-9f4b8ae084c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3696b802980145649ec717ba8abcb02b, user_id=29a1ec39a0e24581b74e77df0a6478d9, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:48.331177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a2b6f-557c-4fd6-8acf-9f4b8ae084c1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:48.345453 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:48.348965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 998/1989] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:48 2022] PUT /identity/v3/projects/3696b802980145649ec717ba8abcb02b/users/29a1ec39a0e24581b74e77df0a6478d9/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:48.359874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87a457-58dc-482e-a7bf-ebd3a2c200ea None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.360173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87a457-58dc-482e-a7bf-ebd3a2c200ea None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.360453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87a457-58dc-482e-a7bf-ebd3a2c200ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.373934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 992/1990] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:48.387771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ed783-58ac-41d5-b278-9069f999887d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:48.389808 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-533ed783-58ac-41d5-b278-9069f999887d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:48.391585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2c87a457-58dc-482e-a7bf-ebd3a2c200ea None None] MFA Rules not processed for user `29a1ec39a0e24581b74e77df0a6478d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:48.392763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2c87a457-58dc-482e-a7bf-ebd3a2c200ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:48.464880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ed783-58ac-41d5-b278-9069f999887d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:48.478656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 999/1991] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:48.489264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87a457-58dc-482e-a7bf-ebd3a2c200ea None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:48.491913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2c87a457-58dc-482e-a7bf-ebd3a2c200ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:48.538301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9821ee1-5bbf-4af7-88ee-1f28bb879533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:48.538967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9821ee1-5bbf-4af7-88ee-1f28bb879533 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.539391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9821ee1-5bbf-4af7-88ee-1f28bb879533 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.539672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9821ee1-5bbf-4af7-88ee-1f28bb879533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.558124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87a457-58dc-482e-a7bf-ebd3a2c200ea None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:48.561985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ad58c-f49e-4753-aa9b-66edecbcaab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:48.562723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad58c-f49e-4753-aa9b-66edecbcaab7 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.563084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad58c-f49e-4753-aa9b-66edecbcaab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.563454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad58c-f49e-4753-aa9b-66edecbcaab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.574792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ad58c-f49e-4753-aa9b-66edecbcaab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:48.576288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ad58c-f49e-4753-aa9b-66edecbcaab7 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:48.586389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1000/1992] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:31:48.611285 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9821ee1-5bbf-4af7-88ee-1f28bb879533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:48.612803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9821ee1-5bbf-4af7-88ee-1f28bb879533 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:48.623267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 993/1993] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:31:48.668578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ad58c-f49e-4753-aa9b-66edecbcaab7 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:48.670048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-664ad58c-f49e-4753-aa9b-66edecbcaab7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:48.690191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:48.755170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ad58c-f49e-4753-aa9b-66edecbcaab7 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:48.760877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:48.761840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.762222 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.762573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] PATH_INFO: `/v3/users/434b2c730c9242edb365636e8f5a7bce` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.763861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=434b2c730c9242edb365636e8f5a7bce)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:48.765859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:48.831990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434b2c730c9242edb365636e8f5a7bce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:48.847105 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434b2c730c9242edb365636e8f5a7bce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:48.863402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434b2c730c9242edb365636e8f5a7bce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:48.864524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434b2c730c9242edb365636e8f5a7bce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:48.865251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434b2c730c9242edb365636e8f5a7bce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:48.884478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1001/1994] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:48 2022] DELETE /identity/v3/users/434b2c730c9242edb365636e8f5a7bce => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:48.887945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb4005-4d85-4281-962e-2e08f57e3b9f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:48.888246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb4005-4d85-4281-962e-2e08f57e3b9f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:48.888678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb4005-4d85-4281-962e-2e08f57e3b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:48.909107 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a3fb4005-4d85-4281-962e-2e08f57e3b9f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:48.910360 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a3fb4005-4d85-4281-962e-2e08f57e3b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:48.977526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:48.992906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 994/1995] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:49.004188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:49.005894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:49.088506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fdbdb-bf07-4fb7-bbe6-575b0bdf8bf4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:49.092599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022223fd-9222-48cb-9b41-a41d754ad790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:49.093893 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022223fd-9222-48cb-9b41-a41d754ad790 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:49.093893 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022223fd-9222-48cb-9b41-a41d754ad790 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:49.093893 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022223fd-9222-48cb-9b41-a41d754ad790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:49.096512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb4005-4d85-4281-962e-2e08f57e3b9f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:49.098112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a3fb4005-4d85-4281-962e-2e08f57e3b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:49.110927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb4005-4d85-4281-962e-2e08f57e3b9f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:49.111327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022223fd-9222-48cb-9b41-a41d754ad790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:49.113022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022223fd-9222-48cb-9b41-a41d754ad790 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:49.118495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:49.119530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:49.119965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:49.120173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] PATH_INFO: `/v3/users/ebbee9b2f89f4f56959cfc5c0a1925c5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:49.121491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebbee9b2f89f4f56959cfc5c0a1925c5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:49.123629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:49.124840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1002/1996] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:49.174118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebbee9b2f89f4f56959cfc5c0a1925c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:49.174864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebbee9b2f89f4f56959cfc5c0a1925c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:49.183576 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebbee9b2f89f4f56959cfc5c0a1925c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:49.197138 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebbee9b2f89f4f56959cfc5c0a1925c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:49.197887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebbee9b2f89f4f56959cfc5c0a1925c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:49.212079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 995/1997] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:49 2022] DELETE /identity/v3/users/ebbee9b2f89f4f56959cfc5c0a1925c5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:49.216646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83906-1623-4ff9-86f7-ab14cc0fb91c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:49.216994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83906-1623-4ff9-86f7-ab14cc0fb91c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:49.217285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83906-1623-4ff9-86f7-ab14cc0fb91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:49.236820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4fe83906-1623-4ff9-86f7-ab14cc0fb91c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:49.238081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4fe83906-1623-4ff9-86f7-ab14cc0fb91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:49.273300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:49.304017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1003/1998] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:49.313605 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:49.315158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:49.378906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6855b1a-5a23-421f-a412-aa2693b3796b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:49.382988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1267b4-c564-4333-a7db-050980ee9b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:49.383694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1267b4-c564-4333-a7db-050980ee9b20 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:49.384012 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1267b4-c564-4333-a7db-050980ee9b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:49.384620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1267b4-c564-4333-a7db-050980ee9b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:49.394303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1267b4-c564-4333-a7db-050980ee9b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:49.395809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1267b4-c564-4333-a7db-050980ee9b20 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:49.406725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 996/1999] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:49.769769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe83906-1623-4ff9-86f7-ab14cc0fb91c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:49.771409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4fe83906-1623-4ff9-86f7-ab14cc0fb91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:49.789886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe83906-1623-4ff9-86f7-ab14cc0fb91c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:49.807627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:49.808118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:49.808408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:49.808977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] PATH_INFO: `/v3/projects/621128a29f214578a4ff5494a218ff23` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:49.814143 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=621128a29f214578a4ff5494a218ff23)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:49.816269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:49.879621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '621128a29f214578a4ff5494a218ff23', 'request_id': 'req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:49.888808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '621128a29f214578a4ff5494a218ff23', 'request_id': 'req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:49.889820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1267b4-c564-4333-a7db-050980ee9b20 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:49.891458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5a1267b4-c564-4333-a7db-050980ee9b20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:49.898328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '621128a29f214578a4ff5494a218ff23', 'request_id': 'req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:49.910623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] The token cache is being invalidate because project 621128a29f214578a4ff5494a218ff23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:49.911188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:49.912270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0bed67a4-dcd5-49d5-8b60-4aa812cdd053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:49.916195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1004/2000] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:49 2022] DELETE /identity/v3/projects/621128a29f214578a4ff5494a218ff23 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:49.916431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1267b4-c564-4333-a7db-050980ee9b20 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:49.921146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20170da5-acd4-44e5-a76e-30982036b020 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:49.921534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20170da5-acd4-44e5-a76e-30982036b020 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:49.921913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20170da5-acd4-44e5-a76e-30982036b020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:49.941288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-20170da5-acd4-44e5-a76e-30982036b020 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:49.942538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-20170da5-acd4-44e5-a76e-30982036b020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:49.979249 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:49.980122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:49.980406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:49.980683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] PATH_INFO: `/v3/projects/6f4dbe3626554145871b7c889317fbce` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:49.981887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f4dbe3626554145871b7c889317fbce)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:49.983854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:49.995963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:50.003703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1005/2001] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:50.013591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8334bc2-0386-443f-aaae-da021850c31f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.013820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8334bc2-0386-443f-aaae-da021850c31f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.014154 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8334bc2-0386-443f-aaae-da021850c31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.033002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b8334bc2-0386-443f-aaae-da021850c31f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:50.033964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f4dbe3626554145871b7c889317fbce', 'request_id': 'req-4ee55521-2671-43fe-9bc2-d874b8b28cf5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.034237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b8334bc2-0386-443f-aaae-da021850c31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:50.040918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f4dbe3626554145871b7c889317fbce', 'request_id': 'req-4ee55521-2671-43fe-9bc2-d874b8b28cf5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.056657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f4dbe3626554145871b7c889317fbce', 'request_id': 'req-4ee55521-2671-43fe-9bc2-d874b8b28cf5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.075837 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] The token cache is being invalidate because project 6f4dbe3626554145871b7c889317fbce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:50.076311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.079141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.081145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 997/2002] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:49 2022] DELETE /identity/v3/projects/6f4dbe3626554145871b7c889317fbce => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:50.084878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:50.086478 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:50.108590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1006/2003] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:50.113228 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8334bc2-0386-443f-aaae-da021850c31f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:50.119152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:50.126504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8334bc2-0386-443f-aaae-da021850c31f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:50.137240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d521a-f5b6-4075-a855-aa20c20616d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:50.137929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d521a-f5b6-4075-a855-aa20c20616d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.138289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d521a-f5b6-4075-a855-aa20c20616d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.138570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d521a-f5b6-4075-a855-aa20c20616d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.139270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d521a-f5b6-4075-a855-aa20c20616d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:50.139585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d521a-f5b6-4075-a855-aa20c20616d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:50.141391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d521a-f5b6-4075-a855-aa20c20616d7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:50.157855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1007/2004] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:31:50.158853 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d521a-f5b6-4075-a855-aa20c20616d7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:50.164565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee55521-2671-43fe-9bc2-d874b8b28cf5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:50.169822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d521a-f5b6-4075-a855-aa20c20616d7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:50.169961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:50.171102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.171310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.171594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] PATH_INFO: `/v3/users/341ee814f7374844abae78f13c3b96cf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.172844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=341ee814f7374844abae78f13c3b96cf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:50.174863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:50.175060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955019a-b331-4bb9-8a64-e26bc7f7615e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:50.175690 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955019a-b331-4bb9-8a64-e26bc7f7615e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.175982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955019a-b331-4bb9-8a64-e26bc7f7615e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.176266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955019a-b331-4bb9-8a64-e26bc7f7615e admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.176966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955019a-b331-4bb9-8a64-e26bc7f7615e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:50.179055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955019a-b331-4bb9-8a64-e26bc7f7615e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:50.213124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1008/2005] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:50 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:31:50.214345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955019a-b331-4bb9-8a64-e26bc7f7615e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:50.233716 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '341ee814f7374844abae78f13c3b96cf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.234965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '341ee814f7374844abae78f13c3b96cf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.247939 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '341ee814f7374844abae78f13c3b96cf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.278043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '341ee814f7374844abae78f13c3b96cf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.279236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '341ee814f7374844abae78f13c3b96cf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.293661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 998/2006] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:50 2022] DELETE /identity/v3/users/341ee814f7374844abae78f13c3b96cf => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:50.307068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955019a-b331-4bb9-8a64-e26bc7f7615e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:50.408397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a8078-e707-486a-9622-7170652dcdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:50.409094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a8078-e707-486a-9622-7170652dcdef admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.409380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a8078-e707-486a-9622-7170652dcdef admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.409660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a8078-e707-486a-9622-7170652dcdef admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.410391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a8078-e707-486a-9622-7170652dcdef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:50.413711 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a8078-e707-486a-9622-7170652dcdef admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:50.459523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:50.461015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:50.463098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1009/2007] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:50 2022] POST /identity/v3/users => generated 280 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:50.488485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d93fe4-a4ce-4420-8075-b3513fcc6c84 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:50.498071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adef60f-0f57-4a3c-85aa-97dfa570762d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:50.498753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adef60f-0f57-4a3c-85aa-97dfa570762d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.499162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adef60f-0f57-4a3c-85aa-97dfa570762d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.499295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adef60f-0f57-4a3c-85aa-97dfa570762d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.500580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adef60f-0f57-4a3c-85aa-97dfa570762d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:50.502331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adef60f-0f57-4a3c-85aa-97dfa570762d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:50.517656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 999/2008] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:50.518605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a8078-e707-486a-9622-7170652dcdef admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:50.535792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a8078-e707-486a-9622-7170652dcdef admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:50.541717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef748b1-9847-4761-9b26-25d177ff82cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:50.542473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef748b1-9847-4761-9b26-25d177ff82cd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.542802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef748b1-9847-4761-9b26-25d177ff82cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.543159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef748b1-9847-4761-9b26-25d177ff82cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.544037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef748b1-9847-4761-9b26-25d177ff82cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:50.545819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef748b1-9847-4761-9b26-25d177ff82cd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:50.556801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1010/2009] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:50.561140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adef60f-0f57-4a3c-85aa-97dfa570762d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:50.574136 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adef60f-0f57-4a3c-85aa-97dfa570762d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:50.581506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226e154-241d-478b-897b-6d8b123cb5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:50.582661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226e154-241d-478b-897b-6d8b123cb5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.583218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226e154-241d-478b-897b-6d8b123cb5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.583469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226e154-241d-478b-897b-6d8b123cb5f7 admin admin] PATH_INFO: `/v3/projects/45339c9a714b44478ccf281a914c5e7d/users/48df5feb22744b9ca79a0f56b119408d/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.588815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:50.605678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226e154-241d-478b-897b-6d8b123cb5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45339c9a714b44478ccf281a914c5e7d, user_id=48df5feb22744b9ca79a0f56b119408d, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:50.608617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226e154-241d-478b-897b-6d8b123cb5f7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:50.623214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1000/2010] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:50 2022] PUT /identity/v3/projects/45339c9a714b44478ccf281a914c5e7d/users/48df5feb22744b9ca79a0f56b119408d/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:50.634473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ca7ff-1e6b-4fe3-94ac-0527def1e29c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.634773 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ca7ff-1e6b-4fe3-94ac-0527def1e29c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.635062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ca7ff-1e6b-4fe3-94ac-0527def1e29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.648844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226e154-241d-478b-897b-6d8b123cb5f7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:50.651362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7226e154-241d-478b-897b-6d8b123cb5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:50.665393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f89ca7ff-1e6b-4fe3-94ac-0527def1e29c None None] MFA Rules not processed for user `48df5feb22744b9ca79a0f56b119408d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:50.666644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f89ca7ff-1e6b-4fe3-94ac-0527def1e29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:50.719122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:50.719961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226e154-241d-478b-897b-6d8b123cb5f7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:50.726715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:50.727890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.728202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.728614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] PATH_INFO: `/v3/projects/0d5e600cf1f6444c8c15d969aedec7b0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.729950 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5e600cf1f6444c8c15d969aedec7b0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:50.731865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:50.744407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1011/2011] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:50 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:50.777850 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ca7ff-1e6b-4fe3-94ac-0527def1e29c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:50.779773 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f89ca7ff-1e6b-4fe3-94ac-0527def1e29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:50.792106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5e600cf1f6444c8c15d969aedec7b0', 'request_id': 'req-6fcbe1f2-08df-47c0-9be9-f81d966494d1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.798062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5e600cf1f6444c8c15d969aedec7b0', 'request_id': 'req-6fcbe1f2-08df-47c0-9be9-f81d966494d1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.805490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5e600cf1f6444c8c15d969aedec7b0', 'request_id': 'req-6fcbe1f2-08df-47c0-9be9-f81d966494d1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.822281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] The token cache is being invalidate because project 0d5e600cf1f6444c8c15d969aedec7b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:50.822475 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.823219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6fcbe1f2-08df-47c0-9be9-f81d966494d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:50.824905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1001/2012] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:50 2022] DELETE /identity/v3/projects/0d5e600cf1f6444c8c15d969aedec7b0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:50.826204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a67c3-afb4-4487-b93f-621085935207 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.826656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a67c3-afb4-4487-b93f-621085935207 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.827127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a67c3-afb4-4487-b93f-621085935207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.848694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-366a67c3-afb4-4487-b93f-621085935207 None None] MFA Rules not processed for user `ca97ac5a572b4c52804a26f5e6b8320b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:50.850232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-366a67c3-afb4-4487-b93f-621085935207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:50.891156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ca7ff-1e6b-4fe3-94ac-0527def1e29c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:50.921190 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1002/2013] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:50.926705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a81f7-03ca-4157-9df1-698f39e1986b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.927067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a81f7-03ca-4157-9df1-698f39e1986b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.927421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a81f7-03ca-4157-9df1-698f39e1986b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.948596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4c6a81f7-03ca-4157-9df1-698f39e1986b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:50.949912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4c6a81f7-03ca-4157-9df1-698f39e1986b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:50.950145 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641c37f-f58c-4688-bc34-b7ed97f9364c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:50.950834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641c37f-f58c-4688-bc34-b7ed97f9364c service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:50.951124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641c37f-f58c-4688-bc34-b7ed97f9364c service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:50.951910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641c37f-f58c-4688-bc34-b7ed97f9364c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:50.954281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5641c37f-f58c-4688-bc34-b7ed97f9364c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:50.968614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:51.040698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641c37f-f58c-4688-bc34-b7ed97f9364c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:51.042381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641c37f-f58c-4688-bc34-b7ed97f9364c service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:51.058243 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1012/2014] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:50 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:51.059305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1003/2015] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:51.068270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946300cb-7eef-4a90-b0a7-24d3af1c9b4f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.069261 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946300cb-7eef-4a90-b0a7-24d3af1c9b4f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.069625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946300cb-7eef-4a90-b0a7-24d3af1c9b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.075098 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a81f7-03ca-4157-9df1-698f39e1986b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:51.076722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4c6a81f7-03ca-4157-9df1-698f39e1986b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:51.090322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a81f7-03ca-4157-9df1-698f39e1986b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:51.091035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-946300cb-7eef-4a90-b0a7-24d3af1c9b4f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:51.092328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-946300cb-7eef-4a90-b0a7-24d3af1c9b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:51.095055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa84ef-913c-4a1e-af7b-7f067aa94b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:51.095932 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa84ef-913c-4a1e-af7b-7f067aa94b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.096225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa84ef-913c-4a1e-af7b-7f067aa94b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.096506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa84ef-913c-4a1e-af7b-7f067aa94b85 admin admin] PATH_INFO: `/v3/users/48df5feb22744b9ca79a0f56b119408d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.097751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa84ef-913c-4a1e-af7b-7f067aa94b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48df5feb22744b9ca79a0f56b119408d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:51.099768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa84ef-913c-4a1e-af7b-7f067aa94b85 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:51.102019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1013/2016] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:51.106819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946300cb-7eef-4a90-b0a7-24d3af1c9b4f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:51.118662 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946300cb-7eef-4a90-b0a7-24d3af1c9b4f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:51.124079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779a898-6bea-4dda-9df5-2603fb2d850e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:51.125067 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779a898-6bea-4dda-9df5-2603fb2d850e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.125532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779a898-6bea-4dda-9df5-2603fb2d850e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.126009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779a898-6bea-4dda-9df5-2603fb2d850e admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.127045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779a898-6bea-4dda-9df5-2603fb2d850e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:51.127675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779a898-6bea-4dda-9df5-2603fb2d850e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:51.129751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779a898-6bea-4dda-9df5-2603fb2d850e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:51.145099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1014/2017] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:31:51.150022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779a898-6bea-4dda-9df5-2603fb2d850e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:51.162056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5faa84ef-913c-4a1e-af7b-7f067aa94b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48df5feb22744b9ca79a0f56b119408d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:51.162741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5faa84ef-913c-4a1e-af7b-7f067aa94b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48df5feb22744b9ca79a0f56b119408d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:51.163889 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779a898-6bea-4dda-9df5-2603fb2d850e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:51.168629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614a26c-6058-4e91-881b-07cc9cb7961f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:51.169298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614a26c-6058-4e91-881b-07cc9cb7961f admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.169830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614a26c-6058-4e91-881b-07cc9cb7961f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.170148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614a26c-6058-4e91-881b-07cc9cb7961f admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.170845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614a26c-6058-4e91-881b-07cc9cb7961f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:51.172734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614a26c-6058-4e91-881b-07cc9cb7961f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:51.173855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5faa84ef-913c-4a1e-af7b-7f067aa94b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48df5feb22744b9ca79a0f56b119408d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:51.189805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5faa84ef-913c-4a1e-af7b-7f067aa94b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48df5feb22744b9ca79a0f56b119408d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:51.190581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5faa84ef-913c-4a1e-af7b-7f067aa94b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48df5feb22744b9ca79a0f56b119408d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:51.206553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1015/2018] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:51 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:51.210643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614a26c-6058-4e91-881b-07cc9cb7961f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:51.212592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5614a26c-6058-4e91-881b-07cc9cb7961f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:51.215791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1004/2019] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:51 2022] DELETE /identity/v3/users/48df5feb22744b9ca79a0f56b119408d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:51.220357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e59a6-3c4a-4b78-bc28-8c7a87a66e93 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.220886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e59a6-3c4a-4b78-bc28-8c7a87a66e93 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.221170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e59a6-3c4a-4b78-bc28-8c7a87a66e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.227080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:51.242250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-343e59a6-3c4a-4b78-bc28-8c7a87a66e93 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:51.243510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-343e59a6-3c4a-4b78-bc28-8c7a87a66e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:51.285048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614a26c-6058-4e91-881b-07cc9cb7961f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:51.314086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1005/2020] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:51.324893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e59a6-3c4a-4b78-bc28-8c7a87a66e93 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:51.326333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-343e59a6-3c4a-4b78-bc28-8c7a87a66e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:51.348923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4985cb-b9d2-4f09-a77b-9682e54b159f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:51.350293 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4985cb-b9d2-4f09-a77b-9682e54b159f admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.350583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4985cb-b9d2-4f09-a77b-9682e54b159f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.351268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4985cb-b9d2-4f09-a77b-9682e54b159f admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.351981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4985cb-b9d2-4f09-a77b-9682e54b159f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:51.354224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4985cb-b9d2-4f09-a77b-9682e54b159f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:51.368899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:51.379752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1016/2021] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:51 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:31:51.383841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4985cb-b9d2-4f09-a77b-9682e54b159f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:51.394743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4985cb-b9d2-4f09-a77b-9682e54b159f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:51.399555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e59a6-3c4a-4b78-bc28-8c7a87a66e93 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:51.403230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db53bd-f57c-4c76-9ac5-e71f45eea487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:51.403914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db53bd-f57c-4c76-9ac5-e71f45eea487 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.404296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db53bd-f57c-4c76-9ac5-e71f45eea487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.404584 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db53bd-f57c-4c76-9ac5-e71f45eea487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.404690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ab839-557e-465b-94ab-2ce5ea883f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:51.405309 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db53bd-f57c-4c76-9ac5-e71f45eea487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:51.405430 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab839-557e-465b-94ab-2ce5ea883f02 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.405624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab839-557e-465b-94ab-2ce5ea883f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.405906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab839-557e-465b-94ab-2ce5ea883f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.406993 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db53bd-f57c-4c76-9ac5-e71f45eea487 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:51.417009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ab839-557e-465b-94ab-2ce5ea883f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:51.419184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ab839-557e-465b-94ab-2ce5ea883f02 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:51.422621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1017/2022] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:51.423499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db53bd-f57c-4c76-9ac5-e71f45eea487 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:51.435392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db53bd-f57c-4c76-9ac5-e71f45eea487 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:51.440548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42672c26-2e11-401d-a68c-48f4041bfa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:51.441045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42672c26-2e11-401d-a68c-48f4041bfa1c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.441334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42672c26-2e11-401d-a68c-48f4041bfa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.441782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42672c26-2e11-401d-a68c-48f4041bfa1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.442654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42672c26-2e11-401d-a68c-48f4041bfa1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:51.444099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1006/2023] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:31:51.444938 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42672c26-2e11-401d-a68c-48f4041bfa1c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:51.459761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1018/2024] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:51.460909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ab839-557e-465b-94ab-2ce5ea883f02 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:51.471657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ab839-557e-465b-94ab-2ce5ea883f02 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:51.477840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe805deb-7da6-45c9-a397-c758e012c1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:51.479140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe805deb-7da6-45c9-a397-c758e012c1d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.479436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe805deb-7da6-45c9-a397-c758e012c1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.479724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe805deb-7da6-45c9-a397-c758e012c1d6 admin admin] PATH_INFO: `/v3/projects/e972c4fe72ee4f4d9656a6abf4ec2297/users/d0f3f769b3574e69b7e6d2f5e7247e61/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.496834 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe805deb-7da6-45c9-a397-c758e012c1d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e972c4fe72ee4f4d9656a6abf4ec2297, user_id=d0f3f769b3574e69b7e6d2f5e7247e61, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:51.499667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe805deb-7da6-45c9-a397-c758e012c1d6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:51.524042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1007/2025] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:51 2022] PUT /identity/v3/projects/e972c4fe72ee4f4d9656a6abf4ec2297/users/d0f3f769b3574e69b7e6d2f5e7247e61/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:51.535338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92199aa0-7db3-4080-9231-7aff8dc49987 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.536128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92199aa0-7db3-4080-9231-7aff8dc49987 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.536528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92199aa0-7db3-4080-9231-7aff8dc49987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.588411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-92199aa0-7db3-4080-9231-7aff8dc49987 None None] MFA Rules not processed for user `d0f3f769b3574e69b7e6d2f5e7247e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:51.591340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-92199aa0-7db3-4080-9231-7aff8dc49987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:51.710640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1019/2026] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:51.715043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe805deb-7da6-45c9-a397-c758e012c1d6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:51.799318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92199aa0-7db3-4080-9231-7aff8dc49987 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:51.801284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-92199aa0-7db3-4080-9231-7aff8dc49987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:51.814663 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe805deb-7da6-45c9-a397-c758e012c1d6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:51.821346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92199aa0-7db3-4080-9231-7aff8dc49987 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:51.827508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:51.829022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.829477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.829764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] PATH_INFO: `/v3/projects/45339c9a714b44478ccf281a914c5e7d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.831703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45339c9a714b44478ccf281a914c5e7d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:51.834223 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:51.839773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6cd80-890a-471d-9f30-319b9ac3e4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:51.842102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6cd80-890a-471d-9f30-319b9ac3e4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:51.843606 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6cd80-890a-471d-9f30-319b9ac3e4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:51.843899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6cd80-890a-471d-9f30-319b9ac3e4f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:51.845369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6cd80-890a-471d-9f30-319b9ac3e4f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:51.846441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:51.850379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6cd80-890a-471d-9f30-319b9ac3e4f2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:51.904359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45339c9a714b44478ccf281a914c5e7d', 'request_id': 'req-c662ea19-73e7-42d2-9207-e3beaa3a0715'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:51.905038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1008/2027] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:51 2022] POST /identity/v3/projects => generated 393 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:51.909844 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6cd80-890a-471d-9f30-319b9ac3e4f2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:51.915208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45339c9a714b44478ccf281a914c5e7d', 'request_id': 'req-c662ea19-73e7-42d2-9207-e3beaa3a0715'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:51.923540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45339c9a714b44478ccf281a914c5e7d', 'request_id': 'req-c662ea19-73e7-42d2-9207-e3beaa3a0715'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:51.937652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] The token cache is being invalidate because project 45339c9a714b44478ccf281a914c5e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:51.937775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:51.938602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:51.940560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1020/2028] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:51 2022] DELETE /identity/v3/projects/45339c9a714b44478ccf281a914c5e7d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:51.996899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6cd80-890a-471d-9f30-319b9ac3e4f2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:52.003349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6b78-8607-41e5-9bed-6373844a3a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:52.004139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6b78-8607-41e5-9bed-6373844a3a96 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:52.004424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6b78-8607-41e5-9bed-6373844a3a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:52.004734 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6b78-8607-41e5-9bed-6373844a3a96 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:52.005622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc6b78-8607-41e5-9bed-6373844a3a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:52.007983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc6b78-8607-41e5-9bed-6373844a3a96 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:52.066538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1009/2029] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:51 2022] POST /identity/v3/users => generated 303 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:52.071054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:52.072815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:52.221014 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662ea19-73e7-42d2-9207-e3beaa3a0715 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:52.228116 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31b630-8e48-4674-8d0e-a414a67ee064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:52.228944 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31b630-8e48-4674-8d0e-a414a67ee064 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:52.229608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31b630-8e48-4674-8d0e-a414a67ee064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:52.230022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31b630-8e48-4674-8d0e-a414a67ee064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:52.230840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31b630-8e48-4674-8d0e-a414a67ee064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:52.232590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31b630-8e48-4674-8d0e-a414a67ee064 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:52.249280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1021/2030] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:52.250623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6b78-8607-41e5-9bed-6373844a3a96 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:52.261358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6b78-8607-41e5-9bed-6373844a3a96 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:52.266038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e620a87-17d9-46c9-849a-3b83a5d0fa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:52.266970 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e620a87-17d9-46c9-849a-3b83a5d0fa2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:52.267284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e620a87-17d9-46c9-849a-3b83a5d0fa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:52.267581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e620a87-17d9-46c9-849a-3b83a5d0fa2a admin admin] PATH_INFO: `/v3/projects/059dc1dc44674afd826a391cc66efba7/users/80ee2ae8aa11450c865af3baecb78fe5/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:52.283865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e620a87-17d9-46c9-849a-3b83a5d0fa2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=059dc1dc44674afd826a391cc66efba7, user_id=80ee2ae8aa11450c865af3baecb78fe5, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:52.287572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e620a87-17d9-46c9-849a-3b83a5d0fa2a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:52.301633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1010/2031] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:52 2022] PUT /identity/v3/projects/059dc1dc44674afd826a391cc66efba7/users/80ee2ae8aa11450c865af3baecb78fe5/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:52.307276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31b630-8e48-4674-8d0e-a414a67ee064 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:52.317812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31b630-8e48-4674-8d0e-a414a67ee064 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:52.379285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a27bda-f9f7-4750-a317-d2386340a8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:52.379974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a27bda-f9f7-4750-a317-d2386340a8bd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:52.380296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a27bda-f9f7-4750-a317-d2386340a8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:52.380632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a27bda-f9f7-4750-a317-d2386340a8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:52.381404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a27bda-f9f7-4750-a317-d2386340a8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:52.383061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a27bda-f9f7-4750-a317-d2386340a8bd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:52.393337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1022/2032] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:52.397971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e620a87-17d9-46c9-849a-3b83a5d0fa2a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:52.403810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:52.410197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e620a87-17d9-46c9-849a-3b83a5d0fa2a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:52.415988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b601f67-a240-4a66-9e16-c99b5132936c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:52.416877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b601f67-a240-4a66-9e16-c99b5132936c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:52.417166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b601f67-a240-4a66-9e16-c99b5132936c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:52.417444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b601f67-a240-4a66-9e16-c99b5132936c admin admin] PATH_INFO: `/v3/projects/059dc1dc44674afd826a391cc66efba7/users/80ee2ae8aa11450c865af3baecb78fe5/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:52.419476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b601f67-a240-4a66-9e16-c99b5132936c admin admin] RBAC: Authorizing `identity:create_grant(project_id=059dc1dc44674afd826a391cc66efba7, user_id=80ee2ae8aa11450c865af3baecb78fe5, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:52.422140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b601f67-a240-4a66-9e16-c99b5132936c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:52.433493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1011/2033] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:52 2022] PUT /identity/v3/projects/059dc1dc44674afd826a391cc66efba7/users/80ee2ae8aa11450c865af3baecb78fe5/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:52.444995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054acace-3f53-4efc-a1dd-831fa9f06685 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:52.445291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054acace-3f53-4efc-a1dd-831fa9f06685 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:52.445581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054acace-3f53-4efc-a1dd-831fa9f06685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:52.473133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-054acace-3f53-4efc-a1dd-831fa9f06685 None None] MFA Rules not processed for user `80ee2ae8aa11450c865af3baecb78fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:52.475052 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-054acace-3f53-4efc-a1dd-831fa9f06685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:52.493480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:52.544435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1023/2034] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:52.555641 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b601f67-a240-4a66-9e16-c99b5132936c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:52.557229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0b601f67-a240-4a66-9e16-c99b5132936c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:52.613820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b601f67-a240-4a66-9e16-c99b5132936c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:52.617583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd03e80-fdc8-44a7-bc4d-a1b076d1cfe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:52.618339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd03e80-fdc8-44a7-bc4d-a1b076d1cfe6 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:52.618627 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd03e80-fdc8-44a7-bc4d-a1b076d1cfe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:52.618907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd03e80-fdc8-44a7-bc4d-a1b076d1cfe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:52.628696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd03e80-fdc8-44a7-bc4d-a1b076d1cfe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:52.630184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd03e80-fdc8-44a7-bc4d-a1b076d1cfe6 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:52.640371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1012/2035] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:52.689155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054acace-3f53-4efc-a1dd-831fa9f06685 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:52.690567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-054acace-3f53-4efc-a1dd-831fa9f06685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:52.702600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054acace-3f53-4efc-a1dd-831fa9f06685 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:52.706305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b27ed0-7e6c-4074-8652-ea13ece18ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:52.706970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b27ed0-7e6c-4074-8652-ea13ece18ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:52.707282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b27ed0-7e6c-4074-8652-ea13ece18ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:52.707553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b27ed0-7e6c-4074-8652-ea13ece18ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:52.709876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-46b27ed0-7e6c-4074-8652-ea13ece18ce1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:52.771711 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b27ed0-7e6c-4074-8652-ea13ece18ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:52.773348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b27ed0-7e6c-4074-8652-ea13ece18ce1 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:52.782788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1024/2036] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:56.326018 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd03e80-fdc8-44a7-bc4d-a1b076d1cfe6 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:56.389677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd03e80-fdc8-44a7-bc4d-a1b076d1cfe6 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:56.394220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:56.395102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:56.395385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:56.395661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] PATH_INFO: `/v3/users/d0f3f769b3574e69b7e6d2f5e7247e61` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:56.397448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f3f769b3574e69b7e6d2f5e7247e61)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:56.399667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:56.448398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f3f769b3574e69b7e6d2f5e7247e61'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:56.449072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f3f769b3574e69b7e6d2f5e7247e61'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:56.455989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f3f769b3574e69b7e6d2f5e7247e61'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:56.468718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f3f769b3574e69b7e6d2f5e7247e61'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:56.469390 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f3f769b3574e69b7e6d2f5e7247e61'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:56.484906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1013/2037] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:56 2022] DELETE /identity/v3/users/d0f3f769b3574e69b7e6d2f5e7247e61 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:56.489803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727229b-2bfd-4313-bb0b-8b1a68d4ca1c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:56.490697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727229b-2bfd-4313-bb0b-8b1a68d4ca1c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:56.491018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727229b-2bfd-4313-bb0b-8b1a68d4ca1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:56.509801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2727229b-2bfd-4313-bb0b-8b1a68d4ca1c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:56.511187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2727229b-2bfd-4313-bb0b-8b1a68d4ca1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:56.577803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1025/2038] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:56.588329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:56.589744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:56.604813 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:56.656424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469cdb4-deac-4510-a511-131bfbdb9463 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:56.660365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656c02d-640e-4f78-9ea7-ae4d7c41617b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:56.661006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656c02d-640e-4f78-9ea7-ae4d7c41617b service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:56.661289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656c02d-640e-4f78-9ea7-ae4d7c41617b service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:56.662155 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656c02d-640e-4f78-9ea7-ae4d7c41617b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:56.672139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656c02d-640e-4f78-9ea7-ae4d7c41617b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:56.673576 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656c02d-640e-4f78-9ea7-ae4d7c41617b service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:56.685823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1014/2039] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:31:56.991118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727229b-2bfd-4313-bb0b-8b1a68d4ca1c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:56.992521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2727229b-2bfd-4313-bb0b-8b1a68d4ca1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:57.011384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727229b-2bfd-4313-bb0b-8b1a68d4ca1c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:57.017050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:57.018093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:57.018379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:57.018655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] PATH_INFO: `/v3/projects/e972c4fe72ee4f4d9656a6abf4ec2297` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:57.019860 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e972c4fe72ee4f4d9656a6abf4ec2297)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:57.022256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:57.071191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e972c4fe72ee4f4d9656a6abf4ec2297', 'request_id': 'req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.072904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:57.079807 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e972c4fe72ee4f4d9656a6abf4ec2297', 'request_id': 'req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.086362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e972c4fe72ee4f4d9656a6abf4ec2297', 'request_id': 'req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.096618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] The token cache is being invalidate because project e972c4fe72ee4f4d9656a6abf4ec2297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:57.096948 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.097591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.099396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1026/2040] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:56 2022] DELETE /identity/v3/projects/e972c4fe72ee4f4d9656a6abf4ec2297 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:57.103398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656c02d-640e-4f78-9ea7-ae4d7c41617b service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:57.112999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656c02d-640e-4f78-9ea7-ae4d7c41617b service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:57.169891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538f8ea-0a65-4d04-9ebe-012d5dbbc2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:57.170799 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538f8ea-0a65-4d04-9ebe-012d5dbbc2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:57.171086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538f8ea-0a65-4d04-9ebe-012d5dbbc2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:57.171392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538f8ea-0a65-4d04-9ebe-012d5dbbc2bb admin admin] PATH_INFO: `/v3/users/80ee2ae8aa11450c865af3baecb78fe5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:57.172578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538f8ea-0a65-4d04-9ebe-012d5dbbc2bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ee2ae8aa11450c865af3baecb78fe5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:57.174586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538f8ea-0a65-4d04-9ebe-012d5dbbc2bb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:57.222142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1538f8ea-0a65-4d04-9ebe-012d5dbbc2bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ee2ae8aa11450c865af3baecb78fe5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.222883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1538f8ea-0a65-4d04-9ebe-012d5dbbc2bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ee2ae8aa11450c865af3baecb78fe5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.230585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1538f8ea-0a65-4d04-9ebe-012d5dbbc2bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ee2ae8aa11450c865af3baecb78fe5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.243752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1538f8ea-0a65-4d04-9ebe-012d5dbbc2bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ee2ae8aa11450c865af3baecb78fe5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.244421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1538f8ea-0a65-4d04-9ebe-012d5dbbc2bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ee2ae8aa11450c865af3baecb78fe5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.257131 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1015/2041] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:57 2022] DELETE /identity/v3/users/80ee2ae8aa11450c865af3baecb78fe5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:57.570258 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:57.571727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:57.587625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8bdc3-9bdd-4625-9f96-0af1a10e883b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:57.645101 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:57.646468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:57.646735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:57.647016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] PATH_INFO: `/v3/projects/059dc1dc44674afd826a391cc66efba7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:57.648208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=059dc1dc44674afd826a391cc66efba7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:57.650444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:57.696524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '059dc1dc44674afd826a391cc66efba7', 'request_id': 'req-5e418512-38e0-4a7e-b76a-373866e5f6f9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.704180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '059dc1dc44674afd826a391cc66efba7', 'request_id': 'req-5e418512-38e0-4a7e-b76a-373866e5f6f9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.711597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '059dc1dc44674afd826a391cc66efba7', 'request_id': 'req-5e418512-38e0-4a7e-b76a-373866e5f6f9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.721657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] The token cache is being invalidate because project 059dc1dc44674afd826a391cc66efba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:31:57.721964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.722596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e418512-38e0-4a7e-b76a-373866e5f6f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:57.725259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1027/2042] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:57 2022] DELETE /identity/v3/projects/059dc1dc44674afd826a391cc66efba7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:57.730192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b6c46-e7ee-4b90-932e-ae0bce8e0c96 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:57.730662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b6c46-e7ee-4b90-932e-ae0bce8e0c96 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:57.730754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b6c46-e7ee-4b90-932e-ae0bce8e0c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:57.748542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2a8b6c46-e7ee-4b90-932e-ae0bce8e0c96 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:57.749889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2a8b6c46-e7ee-4b90-932e-ae0bce8e0c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:57.779032 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:57.811350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1016/2043] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:57.819610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e982f0-6652-4f8e-ac1f-58e3ea6bc258 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:57.819900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e982f0-6652-4f8e-ac1f-58e3ea6bc258 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:57.820183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e982f0-6652-4f8e-ac1f-58e3ea6bc258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:57.839123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-67e982f0-6652-4f8e-ac1f-58e3ea6bc258 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:57.840300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-67e982f0-6652-4f8e-ac1f-58e3ea6bc258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:57.848098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1028/2044] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:57.853163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b6c46-e7ee-4b90-932e-ae0bce8e0c96 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:57.862947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b6c46-e7ee-4b90-932e-ae0bce8e0c96 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:57.867913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad58e49-a211-40d4-b205-3a191097f948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:57.868577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad58e49-a211-40d4-b205-3a191097f948 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:57.868865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad58e49-a211-40d4-b205-3a191097f948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:57.869145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad58e49-a211-40d4-b205-3a191097f948 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:57.869877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad58e49-a211-40d4-b205-3a191097f948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:31:57.870224 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad58e49-a211-40d4-b205-3a191097f948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:57.871826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad58e49-a211-40d4-b205-3a191097f948 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:57.884607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1017/2045] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:31:57.888602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e982f0-6652-4f8e-ac1f-58e3ea6bc258 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:57.898305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e982f0-6652-4f8e-ac1f-58e3ea6bc258 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:57.903098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a508b-f54e-4176-b70d-34844b4411b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:57.903744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a508b-f54e-4176-b70d-34844b4411b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:57.904025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a508b-f54e-4176-b70d-34844b4411b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:57.904304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a508b-f54e-4176-b70d-34844b4411b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:57.904989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a508b-f54e-4176-b70d-34844b4411b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:57.906935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a508b-f54e-4176-b70d-34844b4411b2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:57.933339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1029/2046] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:57 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:57.937057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad58e49-a211-40d4-b205-3a191097f948 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:57.947028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad58e49-a211-40d4-b205-3a191097f948 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:58.002933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ba517-bc9b-490b-92e4-9c1994924e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:58.003595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ba517-bc9b-490b-92e4-9c1994924e95 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.003880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ba517-bc9b-490b-92e4-9c1994924e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.004163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ba517-bc9b-490b-92e4-9c1994924e95 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.004854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ba517-bc9b-490b-92e4-9c1994924e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:58.007777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ba517-bc9b-490b-92e4-9c1994924e95 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:58.034727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1018/2047] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:57 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:58.036297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a508b-f54e-4176-b70d-34844b4411b2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:58.047702 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a508b-f54e-4176-b70d-34844b4411b2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:58.053170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c48d62-e601-4f2f-af75-593d4fa9d3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:58.053861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c48d62-e601-4f2f-af75-593d4fa9d3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.054280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c48d62-e601-4f2f-af75-593d4fa9d3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.054439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c48d62-e601-4f2f-af75-593d4fa9d3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.055191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c48d62-e601-4f2f-af75-593d4fa9d3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:58.056796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c48d62-e601-4f2f-af75-593d4fa9d3e7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:58.071043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1030/2048] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:31:58.074955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ba517-bc9b-490b-92e4-9c1994924e95 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:58.094013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ba517-bc9b-490b-92e4-9c1994924e95 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:58.098953 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150cec3-45b3-4052-8375-6412f1297bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:58.099827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150cec3-45b3-4052-8375-6412f1297bff admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.100230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150cec3-45b3-4052-8375-6412f1297bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.100538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150cec3-45b3-4052-8375-6412f1297bff admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.101497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150cec3-45b3-4052-8375-6412f1297bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:58.103435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150cec3-45b3-4052-8375-6412f1297bff admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:58.117708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1019/2049] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:58.122705 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c48d62-e601-4f2f-af75-593d4fa9d3e7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:58.133772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c48d62-e601-4f2f-af75-593d4fa9d3e7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:58.139814 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285941f6-c5f1-4533-911e-a1a056a01638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:58.141674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285941f6-c5f1-4533-911e-a1a056a01638 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.142485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285941f6-c5f1-4533-911e-a1a056a01638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.143579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285941f6-c5f1-4533-911e-a1a056a01638 admin admin] PATH_INFO: `/v3/projects/bf7017d866ec4d86b1b0bde70d29bf4e/users/fec871e3eabd4ca89f873e579b090df9/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.175909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285941f6-c5f1-4533-911e-a1a056a01638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf7017d866ec4d86b1b0bde70d29bf4e, user_id=fec871e3eabd4ca89f873e579b090df9, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:58.180516 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285941f6-c5f1-4533-911e-a1a056a01638 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:58.194141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1031/2050] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:58 2022] PUT /identity/v3/projects/bf7017d866ec4d86b1b0bde70d29bf4e/users/fec871e3eabd4ca89f873e579b090df9/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:58.204204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524673b6-d8f5-4462-a809-7abd36becc93 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.204498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524673b6-d8f5-4462-a809-7abd36becc93 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.204777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524673b6-d8f5-4462-a809-7abd36becc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.231942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-524673b6-d8f5-4462-a809-7abd36becc93 None None] MFA Rules not processed for user `fec871e3eabd4ca89f873e579b090df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:58.233236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-524673b6-d8f5-4462-a809-7abd36becc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:58.297723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1020/2051] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:58.302837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285941f6-c5f1-4533-911e-a1a056a01638 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:58.308471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:58.370994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285941f6-c5f1-4533-911e-a1a056a01638 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:58.376450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba37250-775e-4f05-ac95-593263b23872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:58.377168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba37250-775e-4f05-ac95-593263b23872 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.377454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba37250-775e-4f05-ac95-593263b23872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.377739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba37250-775e-4f05-ac95-593263b23872 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.378484 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba37250-775e-4f05-ac95-593263b23872 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:58.380696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba37250-775e-4f05-ac95-593263b23872 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:58.407384 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1032/2052] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:31:58 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:31:58.412319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524673b6-d8f5-4462-a809-7abd36becc93 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:58.424467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524673b6-d8f5-4462-a809-7abd36becc93 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:58.451372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:58.486330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b0d00-6935-4fc3-9bd8-aeba643ed927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:58.487044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b0d00-6935-4fc3-9bd8-aeba643ed927 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.487326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b0d00-6935-4fc3-9bd8-aeba643ed927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.487600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b0d00-6935-4fc3-9bd8-aeba643ed927 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.488288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b0d00-6935-4fc3-9bd8-aeba643ed927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:58.491170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b0d00-6935-4fc3-9bd8-aeba643ed927 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:58.516140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1021/2053] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:31:58 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:31:58.517459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba37250-775e-4f05-ac95-593263b23872 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:58.528755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba37250-775e-4f05-ac95-593263b23872 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:58.533499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadb749-bc8d-4520-b942-453c298dd840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:58.534203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadb749-bc8d-4520-b942-453c298dd840 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.534492 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadb749-bc8d-4520-b942-453c298dd840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.534773 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadb749-bc8d-4520-b942-453c298dd840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.535519 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadb749-bc8d-4520-b942-453c298dd840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:58.537457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadb749-bc8d-4520-b942-453c298dd840 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:58.554870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1033/2054] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:58.557088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b0d00-6935-4fc3-9bd8-aeba643ed927 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:58.567374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b0d00-6935-4fc3-9bd8-aeba643ed927 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:58.573252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071dddc-6051-4b42-8f26-a741df5ff2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:58.574240 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071dddc-6051-4b42-8f26-a741df5ff2d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.574532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071dddc-6051-4b42-8f26-a741df5ff2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.574814 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071dddc-6051-4b42-8f26-a741df5ff2d7 admin admin] PATH_INFO: `/v3/projects/d9dd1c0115e04d23880e5b1fd81f6487/users/2ea2d48cf6cd4c7e843acf86ac987ff2/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.590773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071dddc-6051-4b42-8f26-a741df5ff2d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9dd1c0115e04d23880e5b1fd81f6487, user_id=2ea2d48cf6cd4c7e843acf86ac987ff2, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:58.593434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071dddc-6051-4b42-8f26-a741df5ff2d7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:58.607522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1022/2055] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:58 2022] PUT /identity/v3/projects/d9dd1c0115e04d23880e5b1fd81f6487/users/2ea2d48cf6cd4c7e843acf86ac987ff2/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:58.611582 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadb749-bc8d-4520-b942-453c298dd840 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:58.622602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadb749-bc8d-4520-b942-453c298dd840 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:58.684223 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fd768-004d-4773-896d-7331027ce068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:58.684900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fd768-004d-4773-896d-7331027ce068 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.685187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fd768-004d-4773-896d-7331027ce068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.685465 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fd768-004d-4773-896d-7331027ce068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.686246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fd768-004d-4773-896d-7331027ce068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:58.687923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fd768-004d-4773-896d-7331027ce068 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:58.698763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1034/2056] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:31:58.702964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071dddc-6051-4b42-8f26-a741df5ff2d7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:58.713470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071dddc-6051-4b42-8f26-a741df5ff2d7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:58.718201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a13180-9027-477c-a6e5-1f7839adce4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:58.719090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a13180-9027-477c-a6e5-1f7839adce4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.719374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a13180-9027-477c-a6e5-1f7839adce4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.719653 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a13180-9027-477c-a6e5-1f7839adce4a admin admin] PATH_INFO: `/v3/projects/d9dd1c0115e04d23880e5b1fd81f6487/users/2ea2d48cf6cd4c7e843acf86ac987ff2/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.721633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a13180-9027-477c-a6e5-1f7839adce4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9dd1c0115e04d23880e5b1fd81f6487, user_id=2ea2d48cf6cd4c7e843acf86ac987ff2, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:58.725105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a13180-9027-477c-a6e5-1f7839adce4a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:58.735073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1023/2057] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:31:58 2022] PUT /identity/v3/projects/d9dd1c0115e04d23880e5b1fd81f6487/users/2ea2d48cf6cd4c7e843acf86ac987ff2/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:31:58.747060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe8858-3aec-4132-82bc-93778e631a6d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.747356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe8858-3aec-4132-82bc-93778e631a6d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.747668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe8858-3aec-4132-82bc-93778e631a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.780559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-83fe8858-3aec-4132-82bc-93778e631a6d None None] MFA Rules not processed for user `2ea2d48cf6cd4c7e843acf86ac987ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:58.781772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-83fe8858-3aec-4132-82bc-93778e631a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:58.845859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:58.850630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1035/2058] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:31:58.861077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a13180-9027-477c-a6e5-1f7839adce4a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:58.862803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b4a13180-9027-477c-a6e5-1f7839adce4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:58.920095 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a13180-9027-477c-a6e5-1f7839adce4a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:58.923679 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86236261-b854-4c42-9629-32ea7a8a7f86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:58.924332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86236261-b854-4c42-9629-32ea7a8a7f86 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:58.924630 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86236261-b854-4c42-9629-32ea7a8a7f86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:58.924908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86236261-b854-4c42-9629-32ea7a8a7f86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:58.981359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86236261-b854-4c42-9629-32ea7a8a7f86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:58.982847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86236261-b854-4c42-9629-32ea7a8a7f86 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:58.992349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1024/2059] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:59.188080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe8858-3aec-4132-82bc-93778e631a6d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:59.189855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-83fe8858-3aec-4132-82bc-93778e631a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:59.336299 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe8858-3aec-4132-82bc-93778e631a6d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:59.343153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:59.344315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:59.344700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:59.345066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] PATH_INFO: `/v3/users/29a1ec39a0e24581b74e77df0a6478d9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:59.346428 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29a1ec39a0e24581b74e77df0a6478d9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:59.348856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:59.404874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29a1ec39a0e24581b74e77df0a6478d9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:59.418736 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29a1ec39a0e24581b74e77df0a6478d9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:59.425842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29a1ec39a0e24581b74e77df0a6478d9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:59.426555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29a1ec39a0e24581b74e77df0a6478d9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:59.427214 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29a1ec39a0e24581b74e77df0a6478d9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:59.441822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1036/2060] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:31:59 2022] DELETE /identity/v3/users/29a1ec39a0e24581b74e77df0a6478d9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:31:59.446533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e1a3a-c4d1-4691-bf75-af14ed465a29 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:59.446826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e1a3a-c4d1-4691-bf75-af14ed465a29 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:59.447276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e1a3a-c4d1-4691-bf75-af14ed465a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:59.465433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5c9e1a3a-c4d1-4691-bf75-af14ed465a29 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:31:59.466662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c9e1a3a-c4d1-4691-bf75-af14ed465a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:59.499558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:31:59.530971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1025/2061] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:31:59.540882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:59.542349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:59.599641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a85ab-60a0-43ea-b090-7def4d0d8b41 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:59.603456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cad7a8-fcbc-4321-bb52-cbf360550bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:59.604124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cad7a8-fcbc-4321-bb52-cbf360550bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:59.604429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cad7a8-fcbc-4321-bb52-cbf360550bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:59.604708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cad7a8-fcbc-4321-bb52-cbf360550bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:59.614528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cad7a8-fcbc-4321-bb52-cbf360550bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:59.616101 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cad7a8-fcbc-4321-bb52-cbf360550bd3 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:59.625698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1037/2062] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:31:59.911385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e1a3a-c4d1-4691-bf75-af14ed465a29 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:31:59.914207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c9e1a3a-c4d1-4691-bf75-af14ed465a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:31:59.929490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e1a3a-c4d1-4691-bf75-af14ed465a29 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:31:59.934439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:31:59.935294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:31:59.935609 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:31:59.935905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] PATH_INFO: `/v3/projects/3696b802980145649ec717ba8abcb02b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:31:59.937118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3696b802980145649ec717ba8abcb02b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:31:59.939330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:31:59.991289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3696b802980145649ec717ba8abcb02b', 'request_id': 'req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:31:59.996785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3696b802980145649ec717ba8abcb02b', 'request_id': 'req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:00.003056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3696b802980145649ec717ba8abcb02b', 'request_id': 'req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:00.017455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] The token cache is being invalidate because project 3696b802980145649ec717ba8abcb02b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:32:00.017778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:00.018437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bb8dd6fd-fbc2-4a72-a9c1-b1fb968f5609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:00.020016 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1026/2063] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:31:59 2022] DELETE /identity/v3/projects/3696b802980145649ec717ba8abcb02b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:32:00.026221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9071b-e590-451f-be14-eb10bb6f0465 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:00.026516 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9071b-e590-451f-be14-eb10bb6f0465 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:00.026819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9071b-e590-451f-be14-eb10bb6f0465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:00.045132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5da9071b-e590-451f-be14-eb10bb6f0465 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:00.046365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5da9071b-e590-451f-be14-eb10bb6f0465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:00.063507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:00.107528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1038/2064] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:32:00.115004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb6a9e-75c8-4fce-b15c-a7a99b74f9ca None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:00.115436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb6a9e-75c8-4fce-b15c-a7a99b74f9ca None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:00.115723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb6a9e-75c8-4fce-b15c-a7a99b74f9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:00.133770 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-affb6a9e-75c8-4fce-b15c-a7a99b74f9ca None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:00.134997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-affb6a9e-75c8-4fce-b15c-a7a99b74f9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:00.142784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1027/2065] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:00.147366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9071b-e590-451f-be14-eb10bb6f0465 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:00.157134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9071b-e590-451f-be14-eb10bb6f0465 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:00.162343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce6d5b-673d-4906-9c9f-2a1f5a7bd528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:00.163002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce6d5b-673d-4906-9c9f-2a1f5a7bd528 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:00.163289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce6d5b-673d-4906-9c9f-2a1f5a7bd528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:00.163567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce6d5b-673d-4906-9c9f-2a1f5a7bd528 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:00.164295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce6d5b-673d-4906-9c9f-2a1f5a7bd528 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:32:00.164604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce6d5b-673d-4906-9c9f-2a1f5a7bd528 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:00.166211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce6d5b-673d-4906-9c9f-2a1f5a7bd528 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:00.177667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1039/2066] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:32:00.181912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb6a9e-75c8-4fce-b15c-a7a99b74f9ca None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:00.191564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb6a9e-75c8-4fce-b15c-a7a99b74f9ca None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:00.196407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ac5a2-63d2-4255-9fad-0ade4106cbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:00.197068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ac5a2-63d2-4255-9fad-0ade4106cbd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:00.197359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ac5a2-63d2-4255-9fad-0ade4106cbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:00.197640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ac5a2-63d2-4255-9fad-0ade4106cbd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:00.198381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ac5a2-63d2-4255-9fad-0ade4106cbd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:00.200290 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ac5a2-63d2-4255-9fad-0ade4106cbd0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:00.225899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1028/2067] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:32:00 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:32:00.230130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce6d5b-673d-4906-9c9f-2a1f5a7bd528 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:00.240077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce6d5b-673d-4906-9c9f-2a1f5a7bd528 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:00.301478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113e5d0-fe9d-4877-8108-eb508293d321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:00.302260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113e5d0-fe9d-4877-8108-eb508293d321 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:00.302691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113e5d0-fe9d-4877-8108-eb508293d321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:00.302977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113e5d0-fe9d-4877-8108-eb508293d321 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:00.303692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113e5d0-fe9d-4877-8108-eb508293d321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:00.305631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113e5d0-fe9d-4877-8108-eb508293d321 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:00.325457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1040/2068] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:32:00 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:32:00.329553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ac5a2-63d2-4255-9fad-0ade4106cbd0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:00.340915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ac5a2-63d2-4255-9fad-0ade4106cbd0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:00.346599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d808ad-cc44-4379-969a-8a05e29ebc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:00.347371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d808ad-cc44-4379-969a-8a05e29ebc55 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:00.347659 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d808ad-cc44-4379-969a-8a05e29ebc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:00.347938 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d808ad-cc44-4379-969a-8a05e29ebc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:00.348676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d808ad-cc44-4379-969a-8a05e29ebc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:00.350856 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d808ad-cc44-4379-969a-8a05e29ebc55 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:00.367327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1029/2069] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:32:00.368482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113e5d0-fe9d-4877-8108-eb508293d321 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:00.378615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113e5d0-fe9d-4877-8108-eb508293d321 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:00.383452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824c310-4583-43d0-acc7-7224d02fe6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:00.384111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824c310-4583-43d0-acc7-7224d02fe6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:00.384400 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824c310-4583-43d0-acc7-7224d02fe6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:00.384681 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824c310-4583-43d0-acc7-7224d02fe6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:00.385487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824c310-4583-43d0-acc7-7224d02fe6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:00.395995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824c310-4583-43d0-acc7-7224d02fe6b0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:00.409307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1041/2070] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:32:00.410845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d808ad-cc44-4379-969a-8a05e29ebc55 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:00.422887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d808ad-cc44-4379-969a-8a05e29ebc55 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:00.429297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02c7d0-88fc-4bcc-b055-9c5fe2a2b645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:00.430414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02c7d0-88fc-4bcc-b055-9c5fe2a2b645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:00.430803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02c7d0-88fc-4bcc-b055-9c5fe2a2b645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:00.431184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02c7d0-88fc-4bcc-b055-9c5fe2a2b645 admin admin] PATH_INFO: `/v3/projects/5d0ba27e9a7f450c87eb189cdd6e235b/users/5439d1a65154486bb4fdfeeaf8ab4b07/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:00.448817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02c7d0-88fc-4bcc-b055-9c5fe2a2b645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d0ba27e9a7f450c87eb189cdd6e235b, user_id=5439d1a65154486bb4fdfeeaf8ab4b07, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:00.451787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02c7d0-88fc-4bcc-b055-9c5fe2a2b645 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:00.466776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1030/2071] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:32:00 2022] PUT /identity/v3/projects/5d0ba27e9a7f450c87eb189cdd6e235b/users/5439d1a65154486bb4fdfeeaf8ab4b07/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:32:00.471588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee6c43-3729-4455-b720-88479025d495 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:00.471906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee6c43-3729-4455-b720-88479025d495 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:00.472277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee6c43-3729-4455-b720-88479025d495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:00.493396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d4ee6c43-3729-4455-b720-88479025d495 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:00.494799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d4ee6c43-3729-4455-b720-88479025d495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:00.567578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1042/2072] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:00.575272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02c7d0-88fc-4bcc-b055-9c5fe2a2b645 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:00.576957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0c02c7d0-88fc-4bcc-b055-9c5fe2a2b645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:00.635150 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02c7d0-88fc-4bcc-b055-9c5fe2a2b645 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:00.639159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fdef7-d9b4-406a-91d5-57623ceebd0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:00.639940 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fdef7-d9b4-406a-91d5-57623ceebd0e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:00.640219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fdef7-d9b4-406a-91d5-57623ceebd0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:00.640501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fdef7-d9b4-406a-91d5-57623ceebd0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:00.650812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fdef7-d9b4-406a-91d5-57623ceebd0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:00.652250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fdef7-d9b4-406a-91d5-57623ceebd0e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:00.657773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:00.666266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1031/2073] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:32:01.561303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee6c43-3729-4455-b720-88479025d495 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:01.562817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d4ee6c43-3729-4455-b720-88479025d495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:01.583299 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee6c43-3729-4455-b720-88479025d495 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:01.590815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:01.591714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:01.592003 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:01.592287 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] PATH_INFO: `/v3/users/fec871e3eabd4ca89f873e579b090df9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:01.593509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fec871e3eabd4ca89f873e579b090df9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:01.596038 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:01.626694 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:01.675496 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fec871e3eabd4ca89f873e579b090df9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:01.691269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fec871e3eabd4ca89f873e579b090df9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:01.699973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fec871e3eabd4ca89f873e579b090df9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:01.700985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fec871e3eabd4ca89f873e579b090df9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:01.701756 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fec871e3eabd4ca89f873e579b090df9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:01.721036 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1043/2074] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:32:01 2022] DELETE /identity/v3/users/fec871e3eabd4ca89f873e579b090df9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:32:01.727401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ef3ed-fe1e-46e2-afd8-2f75d534a4f7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:01.728331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ef3ed-fe1e-46e2-afd8-2f75d534a4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:01.730214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ef3ed-fe1e-46e2-afd8-2f75d534a4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:01.756698 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0e8ef3ed-fe1e-46e2-afd8-2f75d534a4f7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:01.758483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0e8ef3ed-fe1e-46e2-afd8-2f75d534a4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:01.856260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1032/2075] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:01.868811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:01.871809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:01.957830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84e9c1-861a-4b1e-ae7e-f7322e006732 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:01.963312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea945fd2-2cda-47cc-8b32-a9bf12b51ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:01.963995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea945fd2-2cda-47cc-8b32-a9bf12b51ced service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:01.964284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea945fd2-2cda-47cc-8b32-a9bf12b51ced service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:01.964878 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea945fd2-2cda-47cc-8b32-a9bf12b51ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:01.976330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea945fd2-2cda-47cc-8b32-a9bf12b51ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:01.977479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea945fd2-2cda-47cc-8b32-a9bf12b51ced service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:02.001819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1044/2076] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:32:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:32:02.512297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ef3ed-fe1e-46e2-afd8-2f75d534a4f7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:02.514481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0e8ef3ed-fe1e-46e2-afd8-2f75d534a4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:02.552234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ef3ed-fe1e-46e2-afd8-2f75d534a4f7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:02.568623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:02.569561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:02.569788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:02.570202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] PATH_INFO: `/v3/projects/bf7017d866ec4d86b1b0bde70d29bf4e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:02.571496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf7017d866ec4d86b1b0bde70d29bf4e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:02.573442 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:02.642741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf7017d866ec4d86b1b0bde70d29bf4e', 'request_id': 'req-5c92df11-7ec9-4532-aed8-1b59c7884140'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:02.652652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf7017d866ec4d86b1b0bde70d29bf4e', 'request_id': 'req-5c92df11-7ec9-4532-aed8-1b59c7884140'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:02.659883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf7017d866ec4d86b1b0bde70d29bf4e', 'request_id': 'req-5c92df11-7ec9-4532-aed8-1b59c7884140'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:02.679073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] The token cache is being invalidate because project bf7017d866ec4d86b1b0bde70d29bf4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:32:02.679504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:02.680299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:02.688469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea945fd2-2cda-47cc-8b32-a9bf12b51ced service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:02.689576 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1033/2077] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:32:02 2022] DELETE /identity/v3/projects/bf7017d866ec4d86b1b0bde70d29bf4e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:32:02.706571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea945fd2-2cda-47cc-8b32-a9bf12b51ced service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:02.780870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973c071-9ce0-43de-83c6-246c9fed9cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:02.781704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973c071-9ce0-43de-83c6-246c9fed9cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:02.782060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973c071-9ce0-43de-83c6-246c9fed9cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:02.782319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973c071-9ce0-43de-83c6-246c9fed9cf2 admin admin] PATH_INFO: `/v3/users/2ea2d48cf6cd4c7e843acf86ac987ff2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:02.783568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973c071-9ce0-43de-83c6-246c9fed9cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea2d48cf6cd4c7e843acf86ac987ff2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:02.785549 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973c071-9ce0-43de-83c6-246c9fed9cf2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:02.845810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:02.848278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e973c071-9ce0-43de-83c6-246c9fed9cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea2d48cf6cd4c7e843acf86ac987ff2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:02.868710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e973c071-9ce0-43de-83c6-246c9fed9cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea2d48cf6cd4c7e843acf86ac987ff2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:02.878463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e973c071-9ce0-43de-83c6-246c9fed9cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea2d48cf6cd4c7e843acf86ac987ff2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:02.881201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e973c071-9ce0-43de-83c6-246c9fed9cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea2d48cf6cd4c7e843acf86ac987ff2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:02.882936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e973c071-9ce0-43de-83c6-246c9fed9cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea2d48cf6cd4c7e843acf86ac987ff2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:02.903732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1045/2078] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:32:02 2022] DELETE /identity/v3/users/2ea2d48cf6cd4c7e843acf86ac987ff2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:32:03.605715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:03.607319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:03.636244 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92df11-7ec9-4532-aed8-1b59c7884140 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:03.727711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:03.729329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:03.729622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:03.729897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] PATH_INFO: `/v3/projects/d9dd1c0115e04d23880e5b1fd81f6487` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:03.731707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9dd1c0115e04d23880e5b1fd81f6487)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:03.733130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:03.777237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:03.811067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9dd1c0115e04d23880e5b1fd81f6487', 'request_id': 'req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:03.819141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9dd1c0115e04d23880e5b1fd81f6487', 'request_id': 'req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:03.826012 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9dd1c0115e04d23880e5b1fd81f6487', 'request_id': 'req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:03.840332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] The token cache is being invalidate because project d9dd1c0115e04d23880e5b1fd81f6487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:32:03.840712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:03.841432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1ab378d6-e805-49d4-8b3e-f0e3010e7e48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:03.850984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1034/2079] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:32:03 2022] DELETE /identity/v3/projects/d9dd1c0115e04d23880e5b1fd81f6487 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:32:03.856983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1f7c5-bd2b-4ab7-a050-70ef3b00ecea None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:03.857472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1f7c5-bd2b-4ab7-a050-70ef3b00ecea None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:03.857563 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1f7c5-bd2b-4ab7-a050-70ef3b00ecea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:03.877901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-47b1f7c5-bd2b-4ab7-a050-70ef3b00ecea None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:03.879146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-47b1f7c5-bd2b-4ab7-a050-70ef3b00ecea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:03.950543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1046/2080] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:03.961037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5bed3-752a-422d-8bcf-f545848524a9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:03.961550 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5bed3-752a-422d-8bcf-f545848524a9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:03.961661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5bed3-752a-422d-8bcf-f545848524a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:03.986958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5dd5bed3-752a-422d-8bcf-f545848524a9 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:03.988187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5dd5bed3-752a-422d-8bcf-f545848524a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:04.000008 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1035/2081] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:04.001182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1f7c5-bd2b-4ab7-a050-70ef3b00ecea None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:04.012567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1f7c5-bd2b-4ab7-a050-70ef3b00ecea None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:04.017666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4e035-4681-4691-911e-542587f95541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:04.019185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4e035-4681-4691-911e-542587f95541 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:04.019579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4e035-4681-4691-911e-542587f95541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:04.020067 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4e035-4681-4691-911e-542587f95541 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:04.022204 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4e035-4681-4691-911e-542587f95541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:32:04.022204 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4e035-4681-4691-911e-542587f95541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:04.024713 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4e035-4681-4691-911e-542587f95541 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:04.044038 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1047/2082] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:32:04.045989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5bed3-752a-422d-8bcf-f545848524a9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:04.061918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5bed3-752a-422d-8bcf-f545848524a9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:04.068463 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab303854-6563-4d16-95c4-fe5236d520a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:04.070043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab303854-6563-4d16-95c4-fe5236d520a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:04.070486 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab303854-6563-4d16-95c4-fe5236d520a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:04.071042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab303854-6563-4d16-95c4-fe5236d520a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:04.071980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab303854-6563-4d16-95c4-fe5236d520a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:04.074591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab303854-6563-4d16-95c4-fe5236d520a9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:04.115221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1036/2083] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:32:04 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:32:04.120603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4e035-4681-4691-911e-542587f95541 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:04.133685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4e035-4681-4691-911e-542587f95541 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:04.199988 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e4a6b-7f68-4103-b65c-a1cd429fb767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:04.200685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e4a6b-7f68-4103-b65c-a1cd429fb767 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:04.200969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e4a6b-7f68-4103-b65c-a1cd429fb767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:04.202318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e4a6b-7f68-4103-b65c-a1cd429fb767 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:04.202318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e4a6b-7f68-4103-b65c-a1cd429fb767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:04.204245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e4a6b-7f68-4103-b65c-a1cd429fb767 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:04.232925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1048/2084] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:32:04 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:32:04.237107 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab303854-6563-4d16-95c4-fe5236d520a9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:04.253749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab303854-6563-4d16-95c4-fe5236d520a9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:04.264158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39a84a-6ba5-46b6-900b-1d224ef007d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:04.265458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39a84a-6ba5-46b6-900b-1d224ef007d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:04.265919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39a84a-6ba5-46b6-900b-1d224ef007d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:04.266817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39a84a-6ba5-46b6-900b-1d224ef007d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:04.267880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39a84a-6ba5-46b6-900b-1d224ef007d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:04.271772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39a84a-6ba5-46b6-900b-1d224ef007d5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:04.283874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1037/2085] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:32:04.288016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e4a6b-7f68-4103-b65c-a1cd429fb767 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:04.298187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e4a6b-7f68-4103-b65c-a1cd429fb767 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:04.303716 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e13717-6221-467e-b0df-dab176dac611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:04.304285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e13717-6221-467e-b0df-dab176dac611 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:04.304641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e13717-6221-467e-b0df-dab176dac611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:04.304996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e13717-6221-467e-b0df-dab176dac611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:04.305820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e13717-6221-467e-b0df-dab176dac611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:04.307515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e13717-6221-467e-b0df-dab176dac611 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:04.321794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1049/2086] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:32:04.322942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39a84a-6ba5-46b6-900b-1d224ef007d5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:04.335982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39a84a-6ba5-46b6-900b-1d224ef007d5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:04.341738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b0b76-c205-4db2-972b-ebd482c19a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:04.342682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b0b76-c205-4db2-972b-ebd482c19a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:04.342967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b0b76-c205-4db2-972b-ebd482c19a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:04.343243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b0b76-c205-4db2-972b-ebd482c19a51 admin admin] PATH_INFO: `/v3/projects/780ab2df543d46f1be53f326f73b4602/users/c6e46eb9167f4e12be6214d37f7fb2fd/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:04.368646 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b0b76-c205-4db2-972b-ebd482c19a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780ab2df543d46f1be53f326f73b4602, user_id=c6e46eb9167f4e12be6214d37f7fb2fd, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:04.371887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b0b76-c205-4db2-972b-ebd482c19a51 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:04.389517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1038/2087] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:32:04 2022] PUT /identity/v3/projects/780ab2df543d46f1be53f326f73b4602/users/c6e46eb9167f4e12be6214d37f7fb2fd/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:32:04.400951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174212c-c16f-4cdb-ad6c-03e292f1b0ad None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:04.401246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174212c-c16f-4cdb-ad6c-03e292f1b0ad None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:04.401550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174212c-c16f-4cdb-ad6c-03e292f1b0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:04.431691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f174212c-c16f-4cdb-ad6c-03e292f1b0ad None None] MFA Rules not processed for user `c6e46eb9167f4e12be6214d37f7fb2fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:04.433709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f174212c-c16f-4cdb-ad6c-03e292f1b0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:04.451503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:04.545172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1050/2088] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:04.559237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b0b76-c205-4db2-972b-ebd482c19a51 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:04.562845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e76b0b76-c205-4db2-972b-ebd482c19a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:04.626070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b0b76-c205-4db2-972b-ebd482c19a51 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:04.630710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a0076-ad04-41aa-8d6c-4ecfc46d4d5a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa2c8be3105a415ead65579dc18ba40d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:04.631608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a0076-ad04-41aa-8d6c-4ecfc46d4d5a service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:04.631967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a0076-ad04-41aa-8d6c-4ecfc46d4d5a service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:04.632321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a0076-ad04-41aa-8d6c-4ecfc46d4d5a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:04.646961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a0076-ad04-41aa-8d6c-4ecfc46d4d5a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:04.648648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a0076-ad04-41aa-8d6c-4ecfc46d4d5a service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:04.669925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1039/2089] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:32:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:32:06.015926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f3462-edc5-490c-a401-70b99c6f6fde None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:06.016935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f3462-edc5-490c-a401-70b99c6f6fde None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:06.017386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f3462-edc5-490c-a401-70b99c6f6fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:06.079644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-948f3462-edc5-490c-a401-70b99c6f6fde None None] MFA Rules not processed for user `5439d1a65154486bb4fdfeeaf8ab4b07`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:06.081031 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-948f3462-edc5-490c-a401-70b99c6f6fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:06.208776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1051/2090] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:06.209652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a0076-ad04-41aa-8d6c-4ecfc46d4d5a service glance] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:06.211313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-227a0076-ad04-41aa-8d6c-4ecfc46d4d5a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:06.302129 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:06.320676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a0076-ad04-41aa-8d6c-4ecfc46d4d5a service glance] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:06.328412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc45bc-357a-4039-b9a2-6d19f6a8f546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:06.329394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc45bc-357a-4039-b9a2-6d19f6a8f546 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:06.329989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc45bc-357a-4039-b9a2-6d19f6a8f546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:06.330251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc45bc-357a-4039-b9a2-6d19f6a8f546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:06.331355 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc45bc-357a-4039-b9a2-6d19f6a8f546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:06.334337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc45bc-357a-4039-b9a2-6d19f6a8f546 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:06.366787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1040/2091] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:32:06 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:32:06.370590 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f3462-edc5-490c-a401-70b99c6f6fde None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:06.383920 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f3462-edc5-490c-a401-70b99c6f6fde None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:06.482630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe12a4-9b34-4747-9733-3e559d5f8028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:06.483412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe12a4-9b34-4747-9733-3e559d5f8028 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:06.484044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe12a4-9b34-4747-9733-3e559d5f8028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:06.484546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe12a4-9b34-4747-9733-3e559d5f8028 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:06.485578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe12a4-9b34-4747-9733-3e559d5f8028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:06.487974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe12a4-9b34-4747-9733-3e559d5f8028 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:06.520164 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1052/2092] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:32:06 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:32:06.522211 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc45bc-357a-4039-b9a2-6d19f6a8f546 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:06.533702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc45bc-357a-4039-b9a2-6d19f6a8f546 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:06.539765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd432543-3562-4cfb-b452-cf184d065d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:06.540441 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432543-3562-4cfb-b452-cf184d065d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:06.540726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432543-3562-4cfb-b452-cf184d065d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:06.541003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432543-3562-4cfb-b452-cf184d065d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:06.541732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd432543-3562-4cfb-b452-cf184d065d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:06.543371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd432543-3562-4cfb-b452-cf184d065d24 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:06.561553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1041/2093] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:32:06.565600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe12a4-9b34-4747-9733-3e559d5f8028 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:06.576197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe12a4-9b34-4747-9733-3e559d5f8028 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:06.581101 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5e93a-78ce-45c3-a7f4-982465467d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:06.582092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5e93a-78ce-45c3-a7f4-982465467d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:06.582380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5e93a-78ce-45c3-a7f4-982465467d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:06.582664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5e93a-78ce-45c3-a7f4-982465467d63 admin admin] PATH_INFO: `/v3/projects/94470f28d9c0453bae57cb45691072ba/users/9738703dbbc74b758733f2ea6858b886/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:06.601275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5e93a-78ce-45c3-a7f4-982465467d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94470f28d9c0453bae57cb45691072ba, user_id=9738703dbbc74b758733f2ea6858b886, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:06.604216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5e93a-78ce-45c3-a7f4-982465467d63 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:06.620347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1053/2094] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:32:06 2022] PUT /identity/v3/projects/94470f28d9c0453bae57cb45691072ba/users/9738703dbbc74b758733f2ea6858b886/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:32:06.624387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd432543-3562-4cfb-b452-cf184d065d24 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:06.634723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd432543-3562-4cfb-b452-cf184d065d24 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:06.695760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa20fb-0e82-4a6d-bd22-635c23ec04c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:06.696457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa20fb-0e82-4a6d-bd22-635c23ec04c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:06.696740 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa20fb-0e82-4a6d-bd22-635c23ec04c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:06.697016 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa20fb-0e82-4a6d-bd22-635c23ec04c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:06.697748 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa20fb-0e82-4a6d-bd22-635c23ec04c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:06.699471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa20fb-0e82-4a6d-bd22-635c23ec04c3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:06.709975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1042/2095] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:32:06.714325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5e93a-78ce-45c3-a7f4-982465467d63 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:06.724577 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5e93a-78ce-45c3-a7f4-982465467d63 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:06.729290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c0aed-b7dc-479b-bfcc-abc8c0d39cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:06.730212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c0aed-b7dc-479b-bfcc-abc8c0d39cde admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:06.730492 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c0aed-b7dc-479b-bfcc-abc8c0d39cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:06.730794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c0aed-b7dc-479b-bfcc-abc8c0d39cde admin admin] PATH_INFO: `/v3/projects/94470f28d9c0453bae57cb45691072ba/users/9738703dbbc74b758733f2ea6858b886/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:06.732793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c0aed-b7dc-479b-bfcc-abc8c0d39cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=94470f28d9c0453bae57cb45691072ba, user_id=9738703dbbc74b758733f2ea6858b886, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:06.736303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c0aed-b7dc-479b-bfcc-abc8c0d39cde admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:06.743692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:06.748895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1054/2096] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:32:06 2022] PUT /identity/v3/projects/94470f28d9c0453bae57cb45691072ba/users/9738703dbbc74b758733f2ea6858b886/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:32:12.570120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976c159-1887-4d34-af19-3847bc09c176 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:12.571007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976c159-1887-4d34-af19-3847bc09c176 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:12.571294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976c159-1887-4d34-af19-3847bc09c176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:12.621791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f976c159-1887-4d34-af19-3847bc09c176 None None] MFA Rules not processed for user `9738703dbbc74b758733f2ea6858b886`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:12.623189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f976c159-1887-4d34-af19-3847bc09c176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:12.712019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1043/2097] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:12.726989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c0aed-b7dc-479b-bfcc-abc8c0d39cde admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:12.730406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-be1c0aed-b7dc-479b-bfcc-abc8c0d39cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:12.817734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c0aed-b7dc-479b-bfcc-abc8c0d39cde admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:12.823454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d49aef-9b8d-4747-abba-efcb5d82e9c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:12.824127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d49aef-9b8d-4747-abba-efcb5d82e9c9 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:12.824414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d49aef-9b8d-4747-abba-efcb5d82e9c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:12.824695 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d49aef-9b8d-4747-abba-efcb5d82e9c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:12.886564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d49aef-9b8d-4747-abba-efcb5d82e9c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:12.888187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d49aef-9b8d-4747-abba-efcb5d82e9c9 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:12.899464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1055/2098] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:32:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:32:21.253438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8412c0-badb-4b5e-aa8a-fc047a43c63a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:21.254004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8412c0-badb-4b5e-aa8a-fc047a43c63a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:21.254101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8412c0-badb-4b5e-aa8a-fc047a43c63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:21.275363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2b8412c0-badb-4b5e-aa8a-fc047a43c63a None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:21.278537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2b8412c0-badb-4b5e-aa8a-fc047a43c63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:21.333436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:21.361115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1044/2099] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:32:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:21.409710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d49aef-9b8d-4747-abba-efcb5d82e9c9 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:21.412061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-06d49aef-9b8d-4747-abba-efcb5d82e9c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:21.499414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d49aef-9b8d-4747-abba-efcb5d82e9c9 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:21.505150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b00100-5707-434e-97b3-83fc00d23df9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:21.505810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b00100-5707-434e-97b3-83fc00d23df9 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:21.506130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b00100-5707-434e-97b3-83fc00d23df9 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:21.508368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b00100-5707-434e-97b3-83fc00d23df9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:21.514994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b00100-5707-434e-97b3-83fc00d23df9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:21.516585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b00100-5707-434e-97b3-83fc00d23df9 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:21.525114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1056/2100] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:32:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:32:31.368974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:31.521575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:41.237582 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ba4c8-e8c1-44e0-bb22-93f192cfa6bf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:41.239019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ba4c8-e8c1-44e0-bb22-93f192cfa6bf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:41.239019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ba4c8-e8c1-44e0-bb22-93f192cfa6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:41.258729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-591ba4c8-e8c1-44e0-bb22-93f192cfa6bf None None] MFA Rules not processed for user `5439d1a65154486bb4fdfeeaf8ab4b07`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:41.260271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-591ba4c8-e8c1-44e0-bb22-93f192cfa6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:41.267888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1045/2101] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:48.145672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0270650-8dc6-4c78-90c0-ebc36e35890a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:48.146916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0270650-8dc6-4c78-90c0-ebc36e35890a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:48.147349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0270650-8dc6-4c78-90c0-ebc36e35890a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:48.176636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e0270650-8dc6-4c78-90c0-ebc36e35890a None None] MFA Rules not processed for user `774dd66c674a4955a439afd30354efab`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:48.178262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e0270650-8dc6-4c78-90c0-ebc36e35890a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:48.239641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1057/2102] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:49.291584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9a578-c5ce-44a9-b60b-02034c382b0e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:49.292152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9a578-c5ce-44a9-b60b-02034c382b0e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:49.292152 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9a578-c5ce-44a9-b60b-02034c382b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:49.310718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-72b9a578-c5ce-44a9-b60b-02034c382b0e None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:49.311921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-72b9a578-c5ce-44a9-b60b-02034c382b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:49.321884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1046/2103] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jul 21 06:32:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:49.338430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0270650-8dc6-4c78-90c0-ebc36e35890a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:49.355908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0270650-8dc6-4c78-90c0-ebc36e35890a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:49.365092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ff8ad-f919-4e8e-9a12-f87175026bf7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:49.365758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ff8ad-f919-4e8e-9a12-f87175026bf7 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:49.366080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ff8ad-f919-4e8e-9a12-f87175026bf7 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:49.366377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ff8ad-f919-4e8e-9a12-f87175026bf7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:49.375851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ff8ad-f919-4e8e-9a12-f87175026bf7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:49.378324 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ff8ad-f919-4e8e-9a12-f87175026bf7 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:49.390301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1058/2104] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:32:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:32:54.791809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1910336-c562-4596-8d21-5cb678d2ef99 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:54.792448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1910336-c562-4596-8d21-5cb678d2ef99 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:54.792448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1910336-c562-4596-8d21-5cb678d2ef99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:54.822418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d1910336-c562-4596-8d21-5cb678d2ef99 None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:54.824530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d1910336-c562-4596-8d21-5cb678d2ef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:54.845324 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1047/2105] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:32:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:54.864871 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ff8ad-f919-4e8e-9a12-f87175026bf7 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:54.869516 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1910336-c562-4596-8d21-5cb678d2ef99 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:54.870960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d1910336-c562-4596-8d21-5cb678d2ef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:54.887699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ff8ad-f919-4e8e-9a12-f87175026bf7 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:54.901876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34ad6e-fa7c-4bb2-9c2c-e2c67d281155 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:54.902587 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34ad6e-fa7c-4bb2-9c2c-e2c67d281155 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:54.904514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34ad6e-fa7c-4bb2-9c2c-e2c67d281155 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:54.905182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34ad6e-fa7c-4bb2-9c2c-e2c67d281155 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:54.927747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34ad6e-fa7c-4bb2-9c2c-e2c67d281155 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:54.929849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34ad6e-fa7c-4bb2-9c2c-e2c67d281155 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:54.943337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1059/2106] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:32:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:32:54.967573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1910336-c562-4596-8d21-5cb678d2ef99 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:54.975973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf20439-8331-41ce-ad01-d0b6f08351d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:54.977213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf20439-8331-41ce-ad01-d0b6f08351d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:54.978434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf20439-8331-41ce-ad01-d0b6f08351d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:54.979335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf20439-8331-41ce-ad01-d0b6f08351d1 admin admin] PATH_INFO: `/v3/users/ca97ac5a572b4c52804a26f5e6b8320b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:54.981817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf20439-8331-41ce-ad01-d0b6f08351d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca97ac5a572b4c52804a26f5e6b8320b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:54.985449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf20439-8331-41ce-ad01-d0b6f08351d1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:55.115694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-faf20439-8331-41ce-ad01-d0b6f08351d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca97ac5a572b4c52804a26f5e6b8320b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:55.117335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-faf20439-8331-41ce-ad01-d0b6f08351d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca97ac5a572b4c52804a26f5e6b8320b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:55.131435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-faf20439-8331-41ce-ad01-d0b6f08351d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca97ac5a572b4c52804a26f5e6b8320b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:55.171046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-faf20439-8331-41ce-ad01-d0b6f08351d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca97ac5a572b4c52804a26f5e6b8320b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:55.174921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-faf20439-8331-41ce-ad01-d0b6f08351d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca97ac5a572b4c52804a26f5e6b8320b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:55.222347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1048/2107] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:32:54 2022] DELETE /identity/v3/users/ca97ac5a572b4c52804a26f5e6b8320b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:32:55.498585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34ad6e-fa7c-4bb2-9c2c-e2c67d281155 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:55.512936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34ad6e-fa7c-4bb2-9c2c-e2c67d281155 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:55.624787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:55.625964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:55.626621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:55.627128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] PATH_INFO: `/v3/projects/e612f06f547449ac9e9b95dcd32d5928` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:55.637542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e612f06f547449ac9e9b95dcd32d5928)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:55.639959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:55.740382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e612f06f547449ac9e9b95dcd32d5928', 'request_id': 'req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:55.753752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e612f06f547449ac9e9b95dcd32d5928', 'request_id': 'req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:55.769651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e612f06f547449ac9e9b95dcd32d5928', 'request_id': 'req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:55.787175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] The token cache is being invalidate because project e612f06f547449ac9e9b95dcd32d5928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:32:55.787175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:55.788943 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73d47ad3-3452-4f44-8e6d-9cb7927fac0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:55.791170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1060/2108] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:32:55 2022] DELETE /identity/v3/projects/e612f06f547449ac9e9b95dcd32d5928 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:32:55.796783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdd848-96a1-42bc-b2dc-1e1b78d48066 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:55.797294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdd848-96a1-42bc-b2dc-1e1b78d48066 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:55.797551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdd848-96a1-42bc-b2dc-1e1b78d48066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:55.822660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e0bdd848-96a1-42bc-b2dc-1e1b78d48066 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:55.824358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e0bdd848-96a1-42bc-b2dc-1e1b78d48066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:55.991933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:56.022856 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1049/2109] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:56.024000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41c5d0-b530-45ca-a33d-d74db7a70fd2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:56.024448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41c5d0-b530-45ca-a33d-d74db7a70fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:56.024754 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41c5d0-b530-45ca-a33d-d74db7a70fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:56.060633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0d41c5d0-b530-45ca-a33d-d74db7a70fd2 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:56.062635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0d41c5d0-b530-45ca-a33d-d74db7a70fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:56.077175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1061/2110] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:56.079960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdd848-96a1-42bc-b2dc-1e1b78d48066 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:56.119419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdd848-96a1-42bc-b2dc-1e1b78d48066 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:56.136902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164858e0-cf4f-45a7-b296-6b1aa8c583ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:56.137587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164858e0-cf4f-45a7-b296-6b1aa8c583ce admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:56.137873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164858e0-cf4f-45a7-b296-6b1aa8c583ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:56.138179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164858e0-cf4f-45a7-b296-6b1aa8c583ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:56.138935 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164858e0-cf4f-45a7-b296-6b1aa8c583ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:32:56.139245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164858e0-cf4f-45a7-b296-6b1aa8c583ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:56.140947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164858e0-cf4f-45a7-b296-6b1aa8c583ce admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:56.170881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1050/2111] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:32:56.171947 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41c5d0-b530-45ca-a33d-d74db7a70fd2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:56.177440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:56.188570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41c5d0-b530-45ca-a33d-d74db7a70fd2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:56.201206 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4f789-a770-45ce-b90c-4788834fe820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:56.205092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4f789-a770-45ce-b90c-4788834fe820 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:56.205092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4f789-a770-45ce-b90c-4788834fe820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:56.205092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4f789-a770-45ce-b90c-4788834fe820 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:56.205092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4f789-a770-45ce-b90c-4788834fe820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:56.206917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4f789-a770-45ce-b90c-4788834fe820 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:56.275333 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1062/2112] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:32:56 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:32:56.276746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164858e0-cf4f-45a7-b296-6b1aa8c583ce admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:56.291649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164858e0-cf4f-45a7-b296-6b1aa8c583ce admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:56.490762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f789b-fb1c-4e0e-9bdc-8439d22298c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:56.492090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f789b-fb1c-4e0e-9bdc-8439d22298c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:56.492539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f789b-fb1c-4e0e-9bdc-8439d22298c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:56.492815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f789b-fb1c-4e0e-9bdc-8439d22298c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:56.493977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f789b-fb1c-4e0e-9bdc-8439d22298c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:56.502256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f789b-fb1c-4e0e-9bdc-8439d22298c2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:56.538029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1051/2113] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:32:56 2022] POST /identity/v3/users => generated 286 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:32:56.539822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4f789-a770-45ce-b90c-4788834fe820 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:56.558370 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4f789-a770-45ce-b90c-4788834fe820 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:56.565054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8226eb-5c5b-4240-86ae-1c0a8fc5babf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:56.565955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8226eb-5c5b-4240-86ae-1c0a8fc5babf admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:56.566338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8226eb-5c5b-4240-86ae-1c0a8fc5babf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:56.566737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8226eb-5c5b-4240-86ae-1c0a8fc5babf admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:56.567676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8226eb-5c5b-4240-86ae-1c0a8fc5babf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:56.569688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8226eb-5c5b-4240-86ae-1c0a8fc5babf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:56.586212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1063/2114] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:32:56.587660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f789b-fb1c-4e0e-9bdc-8439d22298c2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:56.602018 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f789b-fb1c-4e0e-9bdc-8439d22298c2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:56.614220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b940cab-9d6e-42b3-bc81-16a418fc4f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:56.615323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b940cab-9d6e-42b3-bc81-16a418fc4f10 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:56.615680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b940cab-9d6e-42b3-bc81-16a418fc4f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:56.616025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b940cab-9d6e-42b3-bc81-16a418fc4f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:56.617359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b940cab-9d6e-42b3-bc81-16a418fc4f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:56.619094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b940cab-9d6e-42b3-bc81-16a418fc4f10 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:56.640657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1052/2115] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:32:56.641820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8226eb-5c5b-4240-86ae-1c0a8fc5babf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:56.654958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8226eb-5c5b-4240-86ae-1c0a8fc5babf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:56.660241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aba8bc-0f8c-4930-a20d-e5dc86af9a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:56.661847 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aba8bc-0f8c-4930-a20d-e5dc86af9a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:56.662362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aba8bc-0f8c-4930-a20d-e5dc86af9a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:56.662803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aba8bc-0f8c-4930-a20d-e5dc86af9a40 admin admin] PATH_INFO: `/v3/projects/72719cf38b294403ac8592bdd806162e/users/de9f3e9b7c3a474490fdd380c411c4da/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:56.702247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aba8bc-0f8c-4930-a20d-e5dc86af9a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72719cf38b294403ac8592bdd806162e, user_id=de9f3e9b7c3a474490fdd380c411c4da, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:56.706610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aba8bc-0f8c-4930-a20d-e5dc86af9a40 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:56.735768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1064/2116] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:32:56 2022] PUT /identity/v3/projects/72719cf38b294403ac8592bdd806162e/users/de9f3e9b7c3a474490fdd380c411c4da/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:32:56.737791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d1ec8-2efc-400f-8091-4702b4fd277a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:56.742467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d1ec8-2efc-400f-8091-4702b4fd277a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:56.742467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d1ec8-2efc-400f-8091-4702b4fd277a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:56.780708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-875d1ec8-2efc-400f-8091-4702b4fd277a None None] MFA Rules not processed for user `de9f3e9b7c3a474490fdd380c411c4da`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:56.782413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-875d1ec8-2efc-400f-8091-4702b4fd277a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:56.872891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1053/2117] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:56.890798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aba8bc-0f8c-4930-a20d-e5dc86af9a40 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:56.892813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-81aba8bc-0f8c-4930-a20d-e5dc86af9a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:57.005474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aba8bc-0f8c-4930-a20d-e5dc86af9a40 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:57.014161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28f531-1889-4c03-bb84-b28ff3b14f9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa2c8be3105a415ead65579dc18ba40d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:57.015100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28f531-1889-4c03-bb84-b28ff3b14f9b service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:57.015760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28f531-1889-4c03-bb84-b28ff3b14f9b service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:57.016189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28f531-1889-4c03-bb84-b28ff3b14f9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:57.031438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28f531-1889-4c03-bb84-b28ff3b14f9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:57.046486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28f531-1889-4c03-bb84-b28ff3b14f9b service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:57.060141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1065/2118] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:32:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:32:58.678736 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d1ec8-2efc-400f-8091-4702b4fd277a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:58.680197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-875d1ec8-2efc-400f-8091-4702b4fd277a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:58.832987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:32:58.848519 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d1ec8-2efc-400f-8091-4702b4fd277a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:58.856129 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:58.857232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:58.857919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:58.858358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] PATH_INFO: `/v3/users/c6e46eb9167f4e12be6214d37f7fb2fd` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:58.860218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e46eb9167f4e12be6214d37f7fb2fd)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:58.862631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:58.943508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e46eb9167f4e12be6214d37f7fb2fd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:58.944277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e46eb9167f4e12be6214d37f7fb2fd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:58.954649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e46eb9167f4e12be6214d37f7fb2fd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:58.981409 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e46eb9167f4e12be6214d37f7fb2fd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:58.984415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e46eb9167f4e12be6214d37f7fb2fd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:32:59.032371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1054/2119] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:32:58 2022] DELETE /identity/v3/users/c6e46eb9167f4e12be6214d37f7fb2fd => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:32:59.032733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea47ca-1b32-425d-919b-9605ef2176f7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:59.033199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea47ca-1b32-425d-919b-9605ef2176f7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:59.033300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea47ca-1b32-425d-919b-9605ef2176f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:59.065563 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-80ea47ca-1b32-425d-919b-9605ef2176f7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:32:59.067704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-80ea47ca-1b32-425d-919b-9605ef2176f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:59.166241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1066/2120] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:32:59.175552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:32:59.177796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:32:59.263943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089c7f2-c70e-4427-89b2-b49f2273e955 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:32:59.275493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012deb2-ad43-4e64-bed9-ee492d0d58c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:32:59.277058 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012deb2-ad43-4e64-bed9-ee492d0d58c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:32:59.277882 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012deb2-ad43-4e64-bed9-ee492d0d58c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:32:59.278476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012deb2-ad43-4e64-bed9-ee492d0d58c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:32:59.293815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012deb2-ad43-4e64-bed9-ee492d0d58c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:32:59.295695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012deb2-ad43-4e64-bed9-ee492d0d58c9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:32:59.311823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1055/2121] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:32:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:00.103328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea47ca-1b32-425d-919b-9605ef2176f7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:00.104883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-80ea47ca-1b32-425d-919b-9605ef2176f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:00.127000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea47ca-1b32-425d-919b-9605ef2176f7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:00.138095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:00.139182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:00.139634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:00.140085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] PATH_INFO: `/v3/projects/780ab2df543d46f1be53f326f73b4602` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:00.141459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] RBAC: Authorizing `identity:delete_project(project_id=780ab2df543d46f1be53f326f73b4602)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:00.143586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:00.227827 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:00.260966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780ab2df543d46f1be53f326f73b4602', 'request_id': 'req-c448d730-3936-4370-99ef-4cd6f894118f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:00.286600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780ab2df543d46f1be53f326f73b4602', 'request_id': 'req-c448d730-3936-4370-99ef-4cd6f894118f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:00.299569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780ab2df543d46f1be53f326f73b4602', 'request_id': 'req-c448d730-3936-4370-99ef-4cd6f894118f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:00.326113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] The token cache is being invalidate because project 780ab2df543d46f1be53f326f73b4602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:33:00.326113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:00.327113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c448d730-3936-4370-99ef-4cd6f894118f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:00.337297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1067/2122] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:33:00 2022] DELETE /identity/v3/projects/780ab2df543d46f1be53f326f73b4602 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:33:00.339001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f744d-6e22-44bd-af57-f26cc1b33b29 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:00.339509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f744d-6e22-44bd-af57-f26cc1b33b29 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:00.339618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f744d-6e22-44bd-af57-f26cc1b33b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:00.361906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-858f744d-6e22-44bd-af57-f26cc1b33b29 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:00.363216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-858f744d-6e22-44bd-af57-f26cc1b33b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:00.462097 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1056/2123] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:00.468093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5199-7cca-4ced-8b16-ccc71b18886a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:00.468508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5199-7cca-4ced-8b16-ccc71b18886a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:00.468905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5199-7cca-4ced-8b16-ccc71b18886a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:00.495910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a5dd5199-7cca-4ced-8b16-ccc71b18886a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:00.497477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a5dd5199-7cca-4ced-8b16-ccc71b18886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:00.515396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1068/2124] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:33:00.516753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f744d-6e22-44bd-af57-f26cc1b33b29 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:00.536588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f744d-6e22-44bd-af57-f26cc1b33b29 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:00.569106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad2e12-5cc6-479e-8c2f-85ef4ed13b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:00.569380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad2e12-5cc6-479e-8c2f-85ef4ed13b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:00.569663 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad2e12-5cc6-479e-8c2f-85ef4ed13b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:00.569957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad2e12-5cc6-479e-8c2f-85ef4ed13b2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:00.570717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad2e12-5cc6-479e-8c2f-85ef4ed13b2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:33:00.571028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad2e12-5cc6-479e-8c2f-85ef4ed13b2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:00.575051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad2e12-5cc6-479e-8c2f-85ef4ed13b2e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:00.603271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1057/2125] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:33:00.604511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd5199-7cca-4ced-8b16-ccc71b18886a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:00.614452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd5199-7cca-4ced-8b16-ccc71b18886a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:00.622641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9bb70-6fdb-47ba-a23a-d9a38de259a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:00.623303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9bb70-6fdb-47ba-a23a-d9a38de259a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:00.623621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9bb70-6fdb-47ba-a23a-d9a38de259a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:00.623865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9bb70-6fdb-47ba-a23a-d9a38de259a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:00.624557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9bb70-6fdb-47ba-a23a-d9a38de259a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:00.626568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9bb70-6fdb-47ba-a23a-d9a38de259a9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:00.682110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1069/2126] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:33:00 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:33:00.686143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad2e12-5cc6-479e-8c2f-85ef4ed13b2e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:00.694927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:00.701460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad2e12-5cc6-479e-8c2f-85ef4ed13b2e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:00.812098 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bbf04-7ae6-40a2-b560-c58ac2d5761a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:00.813141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bbf04-7ae6-40a2-b560-c58ac2d5761a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:00.813644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bbf04-7ae6-40a2-b560-c58ac2d5761a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:00.814032 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bbf04-7ae6-40a2-b560-c58ac2d5761a admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:00.821541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bbf04-7ae6-40a2-b560-c58ac2d5761a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:00.821541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bbf04-7ae6-40a2-b560-c58ac2d5761a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:00.852758 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1058/2127] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:33:00 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:33:00.856738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9bb70-6fdb-47ba-a23a-d9a38de259a9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:00.874896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9bb70-6fdb-47ba-a23a-d9a38de259a9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:00.880979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07151940-f464-410a-8e8c-50c54fc3b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:00.881981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07151940-f464-410a-8e8c-50c54fc3b5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:00.882415 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07151940-f464-410a-8e8c-50c54fc3b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:00.883238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07151940-f464-410a-8e8c-50c54fc3b5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:00.884379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07151940-f464-410a-8e8c-50c54fc3b5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:00.886380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07151940-f464-410a-8e8c-50c54fc3b5ec admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:00.907918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1070/2128] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:33:00.909297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bbf04-7ae6-40a2-b560-c58ac2d5761a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:00.940573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bbf04-7ae6-40a2-b560-c58ac2d5761a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:00.954680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50f4d6-2882-4c51-ac85-6b8d71a5b844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:00.954680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50f4d6-2882-4c51-ac85-6b8d71a5b844 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:00.954680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50f4d6-2882-4c51-ac85-6b8d71a5b844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:00.967944 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50f4d6-2882-4c51-ac85-6b8d71a5b844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:00.980763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50f4d6-2882-4c51-ac85-6b8d71a5b844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:00.991171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50f4d6-2882-4c51-ac85-6b8d71a5b844 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:01.012339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07151940-f464-410a-8e8c-50c54fc3b5ec admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:01.018081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1059/2129] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:33:01.024268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07151940-f464-410a-8e8c-50c54fc3b5ec admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:01.030750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1600375-f78c-4ddc-a003-20cfb01955ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:01.032881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1600375-f78c-4ddc-a003-20cfb01955ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:01.033259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1600375-f78c-4ddc-a003-20cfb01955ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:01.033621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1600375-f78c-4ddc-a003-20cfb01955ec admin admin] PATH_INFO: `/v3/projects/eb38392ff9294ce08fb43f32f6176c4c/users/7425f2b3ad664922961e848dfdd010d3/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:01.072233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1600375-f78c-4ddc-a003-20cfb01955ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb38392ff9294ce08fb43f32f6176c4c, user_id=7425f2b3ad664922961e848dfdd010d3, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:01.073547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1600375-f78c-4ddc-a003-20cfb01955ec admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:01.097620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1071/2130] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:33:01 2022] PUT /identity/v3/projects/eb38392ff9294ce08fb43f32f6176c4c/users/7425f2b3ad664922961e848dfdd010d3/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:33:01.110451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effab805-d06c-4833-8e46-3e99331c3b16 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:01.110877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effab805-d06c-4833-8e46-3e99331c3b16 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:01.111054 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effab805-d06c-4833-8e46-3e99331c3b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:01.152939 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-effab805-d06c-4833-8e46-3e99331c3b16 None None] MFA Rules not processed for user `7425f2b3ad664922961e848dfdd010d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:01.154996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-effab805-d06c-4833-8e46-3e99331c3b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:01.279197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1060/2131] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:01.291418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1600375-f78c-4ddc-a003-20cfb01955ec admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:01.294810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d1600375-f78c-4ddc-a003-20cfb01955ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:01.383200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1600375-f78c-4ddc-a003-20cfb01955ec admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:01.392603 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e943fe2-df96-4be2-b0f5-5e28c8399281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:01.393291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e943fe2-df96-4be2-b0f5-5e28c8399281 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:01.393581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e943fe2-df96-4be2-b0f5-5e28c8399281 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:01.393864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e943fe2-df96-4be2-b0f5-5e28c8399281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:01.405601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e943fe2-df96-4be2-b0f5-5e28c8399281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:01.406653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e943fe2-df96-4be2-b0f5-5e28c8399281 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:01.429514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1072/2132] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:03.126502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872318c-b8b5-4dab-8a31-a552fcccb8b7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:03.127000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872318c-b8b5-4dab-8a31-a552fcccb8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:03.127093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872318c-b8b5-4dab-8a31-a552fcccb8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:03.147773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2872318c-b8b5-4dab-8a31-a552fcccb8b7 None None] MFA Rules not processed for user `7425f2b3ad664922961e848dfdd010d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:03.149259 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2872318c-b8b5-4dab-8a31-a552fcccb8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:03.157199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1061/2133] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:04.402453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff3c99-ae4f-43d7-abfa-e0aafcf7bd92 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:04.403303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff3c99-ae4f-43d7-abfa-e0aafcf7bd92 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:04.403773 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff3c99-ae4f-43d7-abfa-e0aafcf7bd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:04.422692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-71ff3c99-ae4f-43d7-abfa-e0aafcf7bd92 None None] MFA Rules not processed for user `5c54ac39faa84c7db5b9b63360216d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:04.424260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-71ff3c99-ae4f-43d7-abfa-e0aafcf7bd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:04.554199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1073/2134] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:13.142069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:14.525491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:14.934173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770be148-9e76-4476-9d89-502c469e4f70 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:14.934710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770be148-9e76-4476-9d89-502c469e4f70 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:14.934710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770be148-9e76-4476-9d89-502c469e4f70 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:14.936092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1062/2135] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 06:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:33:14.950689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751838-2f1a-4b20-9e36-9bf3f9619ffa None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:14.951255 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751838-2f1a-4b20-9e36-9bf3f9619ffa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:14.951602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751838-2f1a-4b20-9e36-9bf3f9619ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:14.973199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-99751838-2f1a-4b20-9e36-9bf3f9619ffa None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:14.979100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-99751838-2f1a-4b20-9e36-9bf3f9619ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:15.103068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1074/2136] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 06:33:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:15.112074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770be148-9e76-4476-9d89-502c469e4f70 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:15.114665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-770be148-9e76-4476-9d89-502c469e4f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:15.198169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770be148-9e76-4476-9d89-502c469e4f70 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:15.202583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b7db1-67fe-4edf-85e7-aacf83ef1faa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa2c8be3105a415ead65579dc18ba40d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:15.203484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b7db1-67fe-4edf-85e7-aacf83ef1faa service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:15.204085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b7db1-67fe-4edf-85e7-aacf83ef1faa service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:15.204549 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b7db1-67fe-4edf-85e7-aacf83ef1faa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:15.218505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b7db1-67fe-4edf-85e7-aacf83ef1faa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:15.220428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b7db1-67fe-4edf-85e7-aacf83ef1faa service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:15.233506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1063/2137] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:15.337857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63e23f-fd78-47e9-8d2d-b9981f196ab5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:15.337857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63e23f-fd78-47e9-8d2d-b9981f196ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:15.337857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63e23f-fd78-47e9-8d2d-b9981f196ab5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:15.337857 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1075/2138] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 06:33:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:33:15.342264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54a85e-9aa2-4d27-8ed6-18783cbe9a9c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:15.342264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54a85e-9aa2-4d27-8ed6-18783cbe9a9c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:15.342264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54a85e-9aa2-4d27-8ed6-18783cbe9a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:15.362727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8e54a85e-9aa2-4d27-8ed6-18783cbe9a9c None None] MFA Rules not processed for user `1a9c3f34eaeb4b538528ff9050252dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:15.364713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8e54a85e-9aa2-4d27-8ed6-18783cbe9a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:15.440005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1064/2139] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 06:33:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:15.452996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63e23f-fd78-47e9-8d2d-b9981f196ab5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:15.455400 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7b63e23f-fd78-47e9-8d2d-b9981f196ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:15.474259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63e23f-fd78-47e9-8d2d-b9981f196ab5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:15.479045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb037ed-bf37-407f-97b1-58fd86ffb6fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:15.480152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb037ed-bf37-407f-97b1-58fd86ffb6fe service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:15.480667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb037ed-bf37-407f-97b1-58fd86ffb6fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:15.481288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb037ed-bf37-407f-97b1-58fd86ffb6fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:15.492619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb037ed-bf37-407f-97b1-58fd86ffb6fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:15.494748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb037ed-bf37-407f-97b1-58fd86ffb6fe service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:15.514177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1076/2140] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:15.653791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c198687-c933-4042-bc69-d2b60601c27c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:15.654120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c198687-c933-4042-bc69-d2b60601c27c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:15.654399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c198687-c933-4042-bc69-d2b60601c27c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:15.656053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1065/2141] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 06:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 06:33:15.662883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59eebb-280e-4c40-8b39-ae4c345d2dca None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:15.664523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59eebb-280e-4c40-8b39-ae4c345d2dca None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:15.664992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59eebb-280e-4c40-8b39-ae4c345d2dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:15.686315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0e59eebb-280e-4c40-8b39-ae4c345d2dca None None] MFA Rules not processed for user `058f099e6f7545a1b2734dbe5afd2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:15.688052 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0e59eebb-280e-4c40-8b39-ae4c345d2dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:15.706933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1077/2142] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 06:33:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:15.719263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c198687-c933-4042-bc69-d2b60601c27c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:15.721845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c198687-c933-4042-bc69-d2b60601c27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:15.737354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c198687-c933-4042-bc69-d2b60601c27c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:15.742147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04563da-d973-4417-abe3-72156250c5eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:15.742819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04563da-d973-4417-abe3-72156250c5eb service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:15.743103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04563da-d973-4417-abe3-72156250c5eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:15.743382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04563da-d973-4417-abe3-72156250c5eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:15.755684 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04563da-d973-4417-abe3-72156250c5eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:15.757225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04563da-d973-4417-abe3-72156250c5eb service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:15.770681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1066/2143] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:25.012858 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59eebb-280e-4c40-8b39-ae4c345d2dca None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:25.023174 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59eebb-280e-4c40-8b39-ae4c345d2dca None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:25.026794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02aaa83-7c4f-4fca-bea0-c0692cccdb61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:25.027435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02aaa83-7c4f-4fca-bea0-c0692cccdb61 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:25.027718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02aaa83-7c4f-4fca-bea0-c0692cccdb61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:25.027995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02aaa83-7c4f-4fca-bea0-c0692cccdb61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:25.029745 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d02aaa83-7c4f-4fca-bea0-c0692cccdb61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:25.095840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02aaa83-7c4f-4fca-bea0-c0692cccdb61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:25.097591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02aaa83-7c4f-4fca-bea0-c0692cccdb61 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:25.107285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1078/2144] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:25.764910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=55 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:26.567399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5ddc9-abb1-469a-84fb-2ef0ed174210 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:26.567701 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5ddc9-abb1-469a-84fb-2ef0ed174210 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:26.567987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5ddc9-abb1-469a-84fb-2ef0ed174210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:26.601648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1cb5ddc9-abb1-469a-84fb-2ef0ed174210 None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:26.603487 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1cb5ddc9-abb1-469a-84fb-2ef0ed174210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:26.672579 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1067/2145] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:33:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:26.705388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02aaa83-7c4f-4fca-bea0-c0692cccdb61 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:26.709090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d02aaa83-7c4f-4fca-bea0-c0692cccdb61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:26.805751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02aaa83-7c4f-4fca-bea0-c0692cccdb61 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:26.811931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0abd4-0ff3-4be4-8989-a388e334eb53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:26.812810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0abd4-0ff3-4be4-8989-a388e334eb53 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:26.813127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0abd4-0ff3-4be4-8989-a388e334eb53 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:26.813407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0abd4-0ff3-4be4-8989-a388e334eb53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:26.832803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:26.835726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0abd4-0ff3-4be4-8989-a388e334eb53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:26.840002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0abd4-0ff3-4be4-8989-a388e334eb53 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:26.863578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1079/2146] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:33:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:33:26.887383 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5ddc9-abb1-469a-84fb-2ef0ed174210 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:26.889498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1cb5ddc9-abb1-469a-84fb-2ef0ed174210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:27.083162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5ddc9-abb1-469a-84fb-2ef0ed174210 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:27.094594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:27.098308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:27.098821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:27.099394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] PATH_INFO: `/v3/users/de9f3e9b7c3a474490fdd380c411c4da` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:27.103447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de9f3e9b7c3a474490fdd380c411c4da)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:27.109497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:27.231821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de9f3e9b7c3a474490fdd380c411c4da'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:27.233803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de9f3e9b7c3a474490fdd380c411c4da'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:27.255959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de9f3e9b7c3a474490fdd380c411c4da'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:27.291627 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de9f3e9b7c3a474490fdd380c411c4da'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:27.293092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de9f3e9b7c3a474490fdd380c411c4da'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:27.337053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1068/2147] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:33:26 2022] DELETE /identity/v3/users/de9f3e9b7c3a474490fdd380c411c4da => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:33:27.339059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3f2b-6d21-427e-995f-64f6b5b0db26 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:27.339567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3f2b-6d21-427e-995f-64f6b5b0db26 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:27.339861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3f2b-6d21-427e-995f-64f6b5b0db26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:27.360479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-efcf3f2b-6d21-427e-995f-64f6b5b0db26 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:27.361623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-efcf3f2b-6d21-427e-995f-64f6b5b0db26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:27.449982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1080/2148] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:33:27.463648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:27.465364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:27.513422 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:27.570769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013f2c1-77d9-4b5d-94eb-93d566ae98d7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:27.576470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff9b1b-f873-4956-92bc-0955b824d301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:27.577776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff9b1b-f873-4956-92bc-0955b824d301 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:27.578281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff9b1b-f873-4956-92bc-0955b824d301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:27.578718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff9b1b-f873-4956-92bc-0955b824d301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:27.599246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff9b1b-f873-4956-92bc-0955b824d301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:27.602305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff9b1b-f873-4956-92bc-0955b824d301 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:27.626344 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1069/2149] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:28.027514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf3f2b-6d21-427e-995f-64f6b5b0db26 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:28.030289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-efcf3f2b-6d21-427e-995f-64f6b5b0db26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:28.049480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf3f2b-6d21-427e-995f-64f6b5b0db26 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:28.061155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:28.066666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:28.066953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:28.067244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] PATH_INFO: `/v3/projects/72719cf38b294403ac8592bdd806162e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:28.069807 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72719cf38b294403ac8592bdd806162e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:28.071654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:28.129511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72719cf38b294403ac8592bdd806162e', 'request_id': 'req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:28.148706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72719cf38b294403ac8592bdd806162e', 'request_id': 'req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:28.155593 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72719cf38b294403ac8592bdd806162e', 'request_id': 'req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:28.166787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] The token cache is being invalidate because project 72719cf38b294403ac8592bdd806162e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:33:28.167077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:28.168046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-32f9a1bb-9d35-4c99-94ed-0e3df1de44e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:28.169734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1081/2150] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:33:28 2022] DELETE /identity/v3/projects/72719cf38b294403ac8592bdd806162e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:33:28.178326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81936001-ba45-4bb3-bb2d-6bdd414c09f3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:28.178865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81936001-ba45-4bb3-bb2d-6bdd414c09f3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:28.178979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81936001-ba45-4bb3-bb2d-6bdd414c09f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:28.199859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-81936001-ba45-4bb3-bb2d-6bdd414c09f3 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:28.201761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-81936001-ba45-4bb3-bb2d-6bdd414c09f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:28.296203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ae50b-e5ed-4cfb-8a00-6812664e1b37 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:28.296507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ae50b-e5ed-4cfb-8a00-6812664e1b37 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:28.296767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ae50b-e5ed-4cfb-8a00-6812664e1b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:28.298480 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1070/2151] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:28.320547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-db3ae50b-e5ed-4cfb-8a00-6812664e1b37 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:28.321755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-db3ae50b-e5ed-4cfb-8a00-6812664e1b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:28.337388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1082/2152] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:28.338578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81936001-ba45-4bb3-bb2d-6bdd414c09f3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:28.352130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81936001-ba45-4bb3-bb2d-6bdd414c09f3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:28.368497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec08d4-3b91-4b8f-a5d5-9b5c375ac873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:28.369112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec08d4-3b91-4b8f-a5d5-9b5c375ac873 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:28.369409 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec08d4-3b91-4b8f-a5d5-9b5c375ac873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:28.369977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec08d4-3b91-4b8f-a5d5-9b5c375ac873 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:28.370466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec08d4-3b91-4b8f-a5d5-9b5c375ac873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:33:28.370779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec08d4-3b91-4b8f-a5d5-9b5c375ac873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:28.372414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec08d4-3b91-4b8f-a5d5-9b5c375ac873 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:28.385460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1071/2153] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:33:28.389518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ae50b-e5ed-4cfb-8a00-6812664e1b37 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:28.403235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ae50b-e5ed-4cfb-8a00-6812664e1b37 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:28.409204 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639e8ee-33f9-41e8-859d-8d9752c89b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:28.410558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639e8ee-33f9-41e8-859d-8d9752c89b7a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:28.410841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639e8ee-33f9-41e8-859d-8d9752c89b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:28.411117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639e8ee-33f9-41e8-859d-8d9752c89b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:28.411817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639e8ee-33f9-41e8-859d-8d9752c89b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:28.413724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639e8ee-33f9-41e8-859d-8d9752c89b7a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:28.455178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1083/2154] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:33:28 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:33:28.459974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec08d4-3b91-4b8f-a5d5-9b5c375ac873 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:28.479995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec08d4-3b91-4b8f-a5d5-9b5c375ac873 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:28.557137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa4ec3-3951-4d68-b512-87e8046e6756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:28.558009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa4ec3-3951-4d68-b512-87e8046e6756 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:28.558304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa4ec3-3951-4d68-b512-87e8046e6756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:28.558589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa4ec3-3951-4d68-b512-87e8046e6756 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:28.559295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa4ec3-3951-4d68-b512-87e8046e6756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:28.561953 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa4ec3-3951-4d68-b512-87e8046e6756 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:28.598658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639e8ee-33f9-41e8-859d-8d9752c89b7a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:28.599289 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1072/2155] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:33:28 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:33:28.615062 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639e8ee-33f9-41e8-859d-8d9752c89b7a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:28.621476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9c5f5-6c04-4fa8-8c4b-42b108d5bee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:28.622353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9c5f5-6c04-4fa8-8c4b-42b108d5bee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:28.622710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9c5f5-6c04-4fa8-8c4b-42b108d5bee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:28.623263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9c5f5-6c04-4fa8-8c4b-42b108d5bee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:28.624303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9c5f5-6c04-4fa8-8c4b-42b108d5bee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:28.626097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9c5f5-6c04-4fa8-8c4b-42b108d5bee4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:28.642877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1084/2156] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:33:28.648615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa4ec3-3951-4d68-b512-87e8046e6756 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:28.661374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa4ec3-3951-4d68-b512-87e8046e6756 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:28.667106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f0be7-60aa-4338-9d21-8d8633c0a9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:28.668019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f0be7-60aa-4338-9d21-8d8633c0a9be admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:28.668467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f0be7-60aa-4338-9d21-8d8633c0a9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:28.668894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f0be7-60aa-4338-9d21-8d8633c0a9be admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:28.669782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f0be7-60aa-4338-9d21-8d8633c0a9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:28.671578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f0be7-60aa-4338-9d21-8d8633c0a9be admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:28.687801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1073/2157] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:33:28.692209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9c5f5-6c04-4fa8-8c4b-42b108d5bee4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:28.705454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9c5f5-6c04-4fa8-8c4b-42b108d5bee4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:28.711744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1f25-f646-41b5-a9f8-02fbac798d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:28.712945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1f25-f646-41b5-a9f8-02fbac798d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:28.713394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1f25-f646-41b5-a9f8-02fbac798d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:28.713873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1f25-f646-41b5-a9f8-02fbac798d0f admin admin] PATH_INFO: `/v3/projects/fa7834149bfc4fbb8edcd68c37312add/users/6c4d14e64b8c4401bdc54cb7a11c2dc7/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:28.740526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e1f25-f646-41b5-a9f8-02fbac798d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa7834149bfc4fbb8edcd68c37312add, user_id=6c4d14e64b8c4401bdc54cb7a11c2dc7, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:28.743382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e1f25-f646-41b5-a9f8-02fbac798d0f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:28.762577 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1085/2158] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:33:28 2022] PUT /identity/v3/projects/fa7834149bfc4fbb8edcd68c37312add/users/6c4d14e64b8c4401bdc54cb7a11c2dc7/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:33:28.774851 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434a17e-69bc-4956-96ba-187bd82a5996 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:28.775139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434a17e-69bc-4956-96ba-187bd82a5996 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:28.775445 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434a17e-69bc-4956-96ba-187bd82a5996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:28.814436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9434a17e-69bc-4956-96ba-187bd82a5996 None None] MFA Rules not processed for user `6c4d14e64b8c4401bdc54cb7a11c2dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:28.816793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9434a17e-69bc-4956-96ba-187bd82a5996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:28.850584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:28.915336 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1074/2159] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:28.929202 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1f25-f646-41b5-a9f8-02fbac798d0f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:28.931387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-932e1f25-f646-41b5-a9f8-02fbac798d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:28.995794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:29.083630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1f25-f646-41b5-a9f8-02fbac798d0f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:29.088114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086a82e-5376-4c2d-ac85-c61ce131514f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:29.088793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086a82e-5376-4c2d-ac85-c61ce131514f service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:29.089078 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086a82e-5376-4c2d-ac85-c61ce131514f service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:29.089420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086a82e-5376-4c2d-ac85-c61ce131514f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:29.104506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086a82e-5376-4c2d-ac85-c61ce131514f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:29.105540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086a82e-5376-4c2d-ac85-c61ce131514f service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:29.117046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1086/2160] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:29.167567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434a17e-69bc-4956-96ba-187bd82a5996 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:29.294593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434a17e-69bc-4956-96ba-187bd82a5996 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:29.299491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:29.301112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:29.301477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:29.301828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] PATH_INFO: `/v3/users/6c4d14e64b8c4401bdc54cb7a11c2dc7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:29.315049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4d14e64b8c4401bdc54cb7a11c2dc7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:29.317306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:29.404021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4d14e64b8c4401bdc54cb7a11c2dc7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:29.404785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4d14e64b8c4401bdc54cb7a11c2dc7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:29.424545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4d14e64b8c4401bdc54cb7a11c2dc7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:29.439598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4d14e64b8c4401bdc54cb7a11c2dc7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:29.441347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4d14e64b8c4401bdc54cb7a11c2dc7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:29.466106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1075/2161] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:33:29 2022] DELETE /identity/v3/users/6c4d14e64b8c4401bdc54cb7a11c2dc7 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:33:29.470752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d7731-720a-4272-b14b-bf294bc46029 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:29.471052 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d7731-720a-4272-b14b-bf294bc46029 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:29.471339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d7731-720a-4272-b14b-bf294bc46029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:29.494425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2d2d7731-720a-4272-b14b-bf294bc46029 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:29.495631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2d2d7731-720a-4272-b14b-bf294bc46029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:29.635821 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1087/2162] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:29.647968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:29.649522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:29.702332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:29.720223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae7064-a773-4b03-93df-1b6f3080b9a7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:29.725649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb0c1f-8155-482a-a034-8d3f5eef6ad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:29.726371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb0c1f-8155-482a-a034-8d3f5eef6ad5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:29.726661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb0c1f-8155-482a-a034-8d3f5eef6ad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:29.726943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb0c1f-8155-482a-a034-8d3f5eef6ad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:29.737276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb0c1f-8155-482a-a034-8d3f5eef6ad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:29.739674 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb0c1f-8155-482a-a034-8d3f5eef6ad5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:29.751602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1076/2163] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:30.165872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d7731-720a-4272-b14b-bf294bc46029 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:30.167367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2d2d7731-720a-4272-b14b-bf294bc46029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:30.180928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d7731-720a-4272-b14b-bf294bc46029 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:30.187848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:30.189075 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:30.189383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:30.189763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] PATH_INFO: `/v3/projects/fa7834149bfc4fbb8edcd68c37312add` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:30.192097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa7834149bfc4fbb8edcd68c37312add)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:30.195351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:30.289071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa7834149bfc4fbb8edcd68c37312add', 'request_id': 'req-a3b98713-a3df-431c-a110-39eb6fa784d4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:30.300950 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa7834149bfc4fbb8edcd68c37312add', 'request_id': 'req-a3b98713-a3df-431c-a110-39eb6fa784d4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:30.309722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa7834149bfc4fbb8edcd68c37312add', 'request_id': 'req-a3b98713-a3df-431c-a110-39eb6fa784d4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:30.323201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] The token cache is being invalidate because project fa7834149bfc4fbb8edcd68c37312add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:33:30.323574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:30.324534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3b98713-a3df-431c-a110-39eb6fa784d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:30.327006 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1088/2164] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:33:30 2022] DELETE /identity/v3/projects/fa7834149bfc4fbb8edcd68c37312add => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:33:30.334246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53689ce-33fa-48c6-b6f3-42ba71fa6174 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:30.335247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53689ce-33fa-48c6-b6f3-42ba71fa6174 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:30.335734 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53689ce-33fa-48c6-b6f3-42ba71fa6174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:30.370381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d53689ce-33fa-48c6-b6f3-42ba71fa6174 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:30.371908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d53689ce-33fa-48c6-b6f3-42ba71fa6174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:30.442513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1077/2165] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:30.451049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357dece-b540-405f-a0dd-c844985b3595 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:30.451339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357dece-b540-405f-a0dd-c844985b3595 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:30.451621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357dece-b540-405f-a0dd-c844985b3595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:30.471502 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4357dece-b540-405f-a0dd-c844985b3595 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:30.473390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4357dece-b540-405f-a0dd-c844985b3595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:30.481340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1089/2166] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:30.486381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53689ce-33fa-48c6-b6f3-42ba71fa6174 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:30.497787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53689ce-33fa-48c6-b6f3-42ba71fa6174 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:30.502755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c4e0c-2e26-4bc3-9354-a69be58e8bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:30.503586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c4e0c-2e26-4bc3-9354-a69be58e8bfc admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:30.503970 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c4e0c-2e26-4bc3-9354-a69be58e8bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:30.504417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c4e0c-2e26-4bc3-9354-a69be58e8bfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:30.505253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c4e0c-2e26-4bc3-9354-a69be58e8bfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:33:30.505666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c4e0c-2e26-4bc3-9354-a69be58e8bfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:30.507501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c4e0c-2e26-4bc3-9354-a69be58e8bfc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:30.521786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1078/2167] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:33:30.526561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357dece-b540-405f-a0dd-c844985b3595 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:30.536872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357dece-b540-405f-a0dd-c844985b3595 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:30.542040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d865e0-5002-4cd0-ac1b-16547318680f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:30.543164 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d865e0-5002-4cd0-ac1b-16547318680f admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:30.543637 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d865e0-5002-4cd0-ac1b-16547318680f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:30.544053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d865e0-5002-4cd0-ac1b-16547318680f admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:30.545256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d865e0-5002-4cd0-ac1b-16547318680f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:30.548344 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d865e0-5002-4cd0-ac1b-16547318680f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:30.584493 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1090/2168] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:33:30 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:33:30.591241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c4e0c-2e26-4bc3-9354-a69be58e8bfc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:30.610639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c4e0c-2e26-4bc3-9354-a69be58e8bfc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:30.695984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07789c17-dd6e-4a71-b4d6-fca085de3d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:30.696600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789c17-dd6e-4a71-b4d6-fca085de3d32 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:30.696888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789c17-dd6e-4a71-b4d6-fca085de3d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:30.697170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789c17-dd6e-4a71-b4d6-fca085de3d32 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:30.697896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07789c17-dd6e-4a71-b4d6-fca085de3d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:30.701710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07789c17-dd6e-4a71-b4d6-fca085de3d32 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:30.731356 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1079/2169] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:33:30 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:33:30.735231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d865e0-5002-4cd0-ac1b-16547318680f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:30.756949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d865e0-5002-4cd0-ac1b-16547318680f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:30.783600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba158a-51f3-4c10-aca5-35c0d2e7370f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:30.787201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba158a-51f3-4c10-aca5-35c0d2e7370f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:30.787779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba158a-51f3-4c10-aca5-35c0d2e7370f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:30.788749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba158a-51f3-4c10-aca5-35c0d2e7370f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:30.789868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba158a-51f3-4c10-aca5-35c0d2e7370f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:30.791797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba158a-51f3-4c10-aca5-35c0d2e7370f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:30.807320 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1091/2170] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:33:30.811309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07789c17-dd6e-4a71-b4d6-fca085de3d32 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:30.823380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07789c17-dd6e-4a71-b4d6-fca085de3d32 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:30.829134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a72cdb-58b9-4ce7-a542-20941a61f299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:30.829892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a72cdb-58b9-4ce7-a542-20941a61f299 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:30.830281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a72cdb-58b9-4ce7-a542-20941a61f299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:30.830630 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a72cdb-58b9-4ce7-a542-20941a61f299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:30.832010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a72cdb-58b9-4ce7-a542-20941a61f299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:30.833719 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a72cdb-58b9-4ce7-a542-20941a61f299 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:30.853681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1080/2171] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:33:30.863868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba158a-51f3-4c10-aca5-35c0d2e7370f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:30.869350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:30.877476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba158a-51f3-4c10-aca5-35c0d2e7370f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:30.886230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16417d97-f714-4296-97bc-cdb64b0f4818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:30.888539 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16417d97-f714-4296-97bc-cdb64b0f4818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:30.888847 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16417d97-f714-4296-97bc-cdb64b0f4818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:30.889389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16417d97-f714-4296-97bc-cdb64b0f4818 admin admin] PATH_INFO: `/v3/projects/854daeeee974410c9a0afafee795e72c/users/efafaa14b3e146eba88412c382a8b977/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:30.917386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16417d97-f714-4296-97bc-cdb64b0f4818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=854daeeee974410c9a0afafee795e72c, user_id=efafaa14b3e146eba88412c382a8b977, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:30.920832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16417d97-f714-4296-97bc-cdb64b0f4818 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:30.951141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1092/2172] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:33:30 2022] PUT /identity/v3/projects/854daeeee974410c9a0afafee795e72c/users/efafaa14b3e146eba88412c382a8b977/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:33:30.958999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb1685-0385-4052-8220-59b0c456aec8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:30.959356 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb1685-0385-4052-8220-59b0c456aec8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:30.960013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb1685-0385-4052-8220-59b0c456aec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:30.984440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-02eb1685-0385-4052-8220-59b0c456aec8 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:30.986272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-02eb1685-0385-4052-8220-59b0c456aec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:31.037572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:31.065586 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1081/2173] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:31.075843 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16417d97-f714-4296-97bc-cdb64b0f4818 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:31.077644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-16417d97-f714-4296-97bc-cdb64b0f4818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:31.154615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16417d97-f714-4296-97bc-cdb64b0f4818 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:31.161043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc1c8d-1bff-4aa9-a885-9c1eede0b886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:31.161759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc1c8d-1bff-4aa9-a885-9c1eede0b886 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:31.162138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc1c8d-1bff-4aa9-a885-9c1eede0b886 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:31.162418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc1c8d-1bff-4aa9-a885-9c1eede0b886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:31.172934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc1c8d-1bff-4aa9-a885-9c1eede0b886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:31.174484 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc1c8d-1bff-4aa9-a885-9c1eede0b886 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:31.185885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1093/2174] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:32.021486 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9340a-b9b2-435b-acf4-4775304a09aa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:32.022212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9340a-b9b2-435b-acf4-4775304a09aa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:32.022558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9340a-b9b2-435b-acf4-4775304a09aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:32.089280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6aa9340a-b9b2-435b-acf4-4775304a09aa None None] MFA Rules not processed for user `efafaa14b3e146eba88412c382a8b977`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:32.090696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6aa9340a-b9b2-435b-acf4-4775304a09aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:32.208778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1082/2175] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:32.220391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc1c8d-1bff-4aa9-a885-9c1eede0b886 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:32.222550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fddc1c8d-1bff-4aa9-a885-9c1eede0b886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:32.299935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc1c8d-1bff-4aa9-a885-9c1eede0b886 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:32.304602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c417f54-5729-4e35-b260-38e311403594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:32.305518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c417f54-5729-4e35-b260-38e311403594 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:32.305810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c417f54-5729-4e35-b260-38e311403594 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:32.306135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c417f54-5729-4e35-b260-38e311403594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:32.315538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c417f54-5729-4e35-b260-38e311403594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:32.317238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c417f54-5729-4e35-b260-38e311403594 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:32.332652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1094/2176] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:33.772572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1e4a0-9fa3-4f15-bda9-07f08f598374 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:33.773412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1e4a0-9fa3-4f15-bda9-07f08f598374 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:33.773412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1e4a0-9fa3-4f15-bda9-07f08f598374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:33.793481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d9c1e4a0-9fa3-4f15-bda9-07f08f598374 None None] MFA Rules not processed for user `efafaa14b3e146eba88412c382a8b977`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:33.794891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d9c1e4a0-9fa3-4f15-bda9-07f08f598374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:33.803265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1083/2177] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:40.435306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c417f54-5729-4e35-b260-38e311403594 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:40.436724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6c417f54-5729-4e35-b260-38e311403594 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:40.510145 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c417f54-5729-4e35-b260-38e311403594 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:40.515539 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7765418-5757-4dec-9556-19b762e51af8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da2d85eb18049f196aeefc51d3f1e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:40.516207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7765418-5757-4dec-9556-19b762e51af8 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:40.516491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7765418-5757-4dec-9556-19b762e51af8 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:40.516772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7765418-5757-4dec-9556-19b762e51af8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:40.518575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e7765418-5757-4dec-9556-19b762e51af8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:40.531089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7765418-5757-4dec-9556-19b762e51af8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:40.532668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7765418-5757-4dec-9556-19b762e51af8 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:40.543057 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1095/2178] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:43.276983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4368638-a0c5-4f1c-ae54-f5f7a72d090a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:43.277517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4368638-a0c5-4f1c-ae54-f5f7a72d090a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:43.277609 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4368638-a0c5-4f1c-ae54-f5f7a72d090a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:43.297507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a4368638-a0c5-4f1c-ae54-f5f7a72d090a None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:43.298885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a4368638-a0c5-4f1c-ae54-f5f7a72d090a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:43.367086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1084/2179] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:33:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:43.381348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7765418-5757-4dec-9556-19b762e51af8 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:43.383126 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e7765418-5757-4dec-9556-19b762e51af8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:43.398701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:43.474723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7765418-5757-4dec-9556-19b762e51af8 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:43.480094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52005f-0b1f-4ffe-a8d4-4cc392d71324 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:43.480875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52005f-0b1f-4ffe-a8d4-4cc392d71324 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:43.481267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52005f-0b1f-4ffe-a8d4-4cc392d71324 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:43.481645 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52005f-0b1f-4ffe-a8d4-4cc392d71324 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:43.491399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52005f-0b1f-4ffe-a8d4-4cc392d71324 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:43.493973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52005f-0b1f-4ffe-a8d4-4cc392d71324 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:43.502002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1096/2180] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:33:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:33:44.036987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c03d8d-6de5-41bf-8f23-4d3374272619 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:44.037286 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c03d8d-6de5-41bf-8f23-4d3374272619 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:44.037603 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c03d8d-6de5-41bf-8f23-4d3374272619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:44.057958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-97c03d8d-6de5-41bf-8f23-4d3374272619 None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:44.059578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-97c03d8d-6de5-41bf-8f23-4d3374272619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:44.067552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1085/2181] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jul 21 06:33:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:33:44.080945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52005f-0b1f-4ffe-a8d4-4cc392d71324 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:44.092902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52005f-0b1f-4ffe-a8d4-4cc392d71324 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:44.100452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac9373-54d6-48e4-9811-52b74d62a958 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:44.101125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac9373-54d6-48e4-9811-52b74d62a958 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:44.101412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac9373-54d6-48e4-9811-52b74d62a958 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:44.101692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac9373-54d6-48e4-9811-52b74d62a958 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:44.110823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac9373-54d6-48e4-9811-52b74d62a958 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:44.113124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac9373-54d6-48e4-9811-52b74d62a958 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:44.127659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1097/2182] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:33:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:33:44.360321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af8e74-e147-4bce-85e0-372eb1d2b045 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:44.361030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af8e74-e147-4bce-85e0-372eb1d2b045 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:44.361570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af8e74-e147-4bce-85e0-372eb1d2b045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:44.383251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-87af8e74-e147-4bce-85e0-372eb1d2b045 None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:44.384532 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-87af8e74-e147-4bce-85e0-372eb1d2b045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:44.393084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1086/2183] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:33:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:44.406156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac9373-54d6-48e4-9811-52b74d62a958 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:44.418464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac9373-54d6-48e4-9811-52b74d62a958 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:44.426903 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a465dc-0a7d-420a-830c-4bfe44eb6d75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:44.427693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a465dc-0a7d-420a-830c-4bfe44eb6d75 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:44.428654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a465dc-0a7d-420a-830c-4bfe44eb6d75 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:44.428654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a465dc-0a7d-420a-830c-4bfe44eb6d75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:44.441242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a465dc-0a7d-420a-830c-4bfe44eb6d75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:44.443984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a465dc-0a7d-420a-830c-4bfe44eb6d75 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:44.454298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1098/2184] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:33:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:33:44.967488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a0ef3-88f9-4c1b-aa7a-fcd7ad581da1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:44.967812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a0ef3-88f9-4c1b-aa7a-fcd7ad581da1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:44.968123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a0ef3-88f9-4c1b-aa7a-fcd7ad581da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:44.994950 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-870a0ef3-88f9-4c1b-aa7a-fcd7ad581da1 None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:44.997154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-870a0ef3-88f9-4c1b-aa7a-fcd7ad581da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:45.006661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1087/2185] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jul 21 06:33:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:45.023617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a465dc-0a7d-420a-830c-4bfe44eb6d75 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:45.034851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a465dc-0a7d-420a-830c-4bfe44eb6d75 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:45.045768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a07c76-081c-490c-9377-28a22868297e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:45.046876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a07c76-081c-490c-9377-28a22868297e service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:45.047349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a07c76-081c-490c-9377-28a22868297e service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:45.047811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a07c76-081c-490c-9377-28a22868297e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:45.062494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a07c76-081c-490c-9377-28a22868297e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:45.064083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a07c76-081c-490c-9377-28a22868297e service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:45.080740 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1099/2186] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:33:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:33:47.317377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a0ef3-88f9-4c1b-aa7a-fcd7ad581da1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:47.318846 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-870a0ef3-88f9-4c1b-aa7a-fcd7ad581da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:47.333030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a0ef3-88f9-4c1b-aa7a-fcd7ad581da1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:47.337646 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:47.338554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:47.338851 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:47.339128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] PATH_INFO: `/v3/users/7425f2b3ad664922961e848dfdd010d3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:47.340350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7425f2b3ad664922961e848dfdd010d3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:47.342369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:47.393654 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7425f2b3ad664922961e848dfdd010d3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:47.394460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7425f2b3ad664922961e848dfdd010d3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:47.401151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:47.403247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7425f2b3ad664922961e848dfdd010d3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:47.418259 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7425f2b3ad664922961e848dfdd010d3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:47.418960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7425f2b3ad664922961e848dfdd010d3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:47.438017 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1088/2187] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:33:47 2022] DELETE /identity/v3/users/7425f2b3ad664922961e848dfdd010d3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:33:47.441697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13001ce7-4fe7-4012-bd69-f02065a09b96 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:47.442159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13001ce7-4fe7-4012-bd69-f02065a09b96 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:47.442254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13001ce7-4fe7-4012-bd69-f02065a09b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:47.460524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-13001ce7-4fe7-4012-bd69-f02065a09b96 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:47.461722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-13001ce7-4fe7-4012-bd69-f02065a09b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:47.520778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1100/2188] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:47.530572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:47.532067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:47.589451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18f44c-69c3-4b58-917d-b47f76e41a1b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:47.593130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ee034-dd97-4792-aee5-668f38d10fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:47.593783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ee034-dd97-4792-aee5-668f38d10fdc service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:47.594163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ee034-dd97-4792-aee5-668f38d10fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:47.594451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ee034-dd97-4792-aee5-668f38d10fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:47.604136 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ee034-dd97-4792-aee5-668f38d10fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:47.605577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ee034-dd97-4792-aee5-668f38d10fdc service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:47.615206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1089/2189] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:47.780421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13001ce7-4fe7-4012-bd69-f02065a09b96 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:47.781832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-13001ce7-4fe7-4012-bd69-f02065a09b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:47.795142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13001ce7-4fe7-4012-bd69-f02065a09b96 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:47.800315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:47.801337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:47.801619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:47.801925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] PATH_INFO: `/v3/projects/eb38392ff9294ce08fb43f32f6176c4c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:47.803143 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb38392ff9294ce08fb43f32f6176c4c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:47.805117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:47.855260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb38392ff9294ce08fb43f32f6176c4c', 'request_id': 'req-8868562b-4db6-4796-a0eb-dde242e66fff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:47.862953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb38392ff9294ce08fb43f32f6176c4c', 'request_id': 'req-8868562b-4db6-4796-a0eb-dde242e66fff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:47.865165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:47.870181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb38392ff9294ce08fb43f32f6176c4c', 'request_id': 'req-8868562b-4db6-4796-a0eb-dde242e66fff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:47.882631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] The token cache is being invalidate because project eb38392ff9294ce08fb43f32f6176c4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:33:47.882919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:47.883594 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8868562b-4db6-4796-a0eb-dde242e66fff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:33:47.885266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1101/2190] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:33:47 2022] DELETE /identity/v3/projects/eb38392ff9294ce08fb43f32f6176c4c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:33:47.890807 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95504081-794f-4ac5-862a-38bc453d6ccc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:47.891146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95504081-794f-4ac5-862a-38bc453d6ccc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:47.891465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95504081-794f-4ac5-862a-38bc453d6ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:47.910171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-95504081-794f-4ac5-862a-38bc453d6ccc None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:47.911366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-95504081-794f-4ac5-862a-38bc453d6ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:47.971102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1090/2191] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:47.979621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c44d6-d468-442f-bcc9-e269fd949ea7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:47.979918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c44d6-d468-442f-bcc9-e269fd949ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:47.980203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c44d6-d468-442f-bcc9-e269fd949ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:47.998380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3c7c44d6-d468-442f-bcc9-e269fd949ea7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:47.999564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3c7c44d6-d468-442f-bcc9-e269fd949ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:48.008070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1102/2192] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:48.012167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95504081-794f-4ac5-862a-38bc453d6ccc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:48.022858 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95504081-794f-4ac5-862a-38bc453d6ccc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:48.027426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a361a0-5ab1-4454-ad34-0915eb93788e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:48.028078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a361a0-5ab1-4454-ad34-0915eb93788e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:48.028364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a361a0-5ab1-4454-ad34-0915eb93788e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:48.028643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a361a0-5ab1-4454-ad34-0915eb93788e admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:48.029405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a361a0-5ab1-4454-ad34-0915eb93788e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:33:48.029712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a361a0-5ab1-4454-ad34-0915eb93788e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:48.031348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a361a0-5ab1-4454-ad34-0915eb93788e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:48.042897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1091/2193] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:33:48.047151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c44d6-d468-442f-bcc9-e269fd949ea7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:48.056692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c44d6-d468-442f-bcc9-e269fd949ea7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:48.061346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831368e9-c388-48f8-811b-c51a43892bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:48.062042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831368e9-c388-48f8-811b-c51a43892bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:48.062333 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831368e9-c388-48f8-811b-c51a43892bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:48.062616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831368e9-c388-48f8-811b-c51a43892bc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:48.063302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831368e9-c388-48f8-811b-c51a43892bc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:48.065208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831368e9-c388-48f8-811b-c51a43892bc2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:48.091764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1103/2194] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:33:48 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:33:48.095898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a361a0-5ab1-4454-ad34-0915eb93788e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:48.106428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a361a0-5ab1-4454-ad34-0915eb93788e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:48.163838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3ac6c-2404-49e1-a7f4-f69dc6697545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:48.164496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3ac6c-2404-49e1-a7f4-f69dc6697545 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:48.164778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3ac6c-2404-49e1-a7f4-f69dc6697545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:48.165053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3ac6c-2404-49e1-a7f4-f69dc6697545 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:48.165742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3ac6c-2404-49e1-a7f4-f69dc6697545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:48.167693 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3ac6c-2404-49e1-a7f4-f69dc6697545 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:48.175922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:48.188485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1092/2195] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:33:48 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:33:48.192628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831368e9-c388-48f8-811b-c51a43892bc2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:48.202789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831368e9-c388-48f8-811b-c51a43892bc2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:48.207846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9e595-d9f1-474e-a766-359a8df37069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:48.208508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9e595-d9f1-474e-a766-359a8df37069 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:48.208796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9e595-d9f1-474e-a766-359a8df37069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:48.209101 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9e595-d9f1-474e-a766-359a8df37069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:48.209821 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9e595-d9f1-474e-a766-359a8df37069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:48.211454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9e595-d9f1-474e-a766-359a8df37069 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:48.223166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1104/2196] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:33:48.226940 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3ac6c-2404-49e1-a7f4-f69dc6697545 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:48.236899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3ac6c-2404-49e1-a7f4-f69dc6697545 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:48.242278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6444bad-edc0-4b58-83ed-780192450fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:48.242935 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6444bad-edc0-4b58-83ed-780192450fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:48.243224 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6444bad-edc0-4b58-83ed-780192450fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:48.243503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6444bad-edc0-4b58-83ed-780192450fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:48.244234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6444bad-edc0-4b58-83ed-780192450fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:48.245920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6444bad-edc0-4b58-83ed-780192450fa7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:48.255835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1093/2197] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:33:48.259779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9e595-d9f1-474e-a766-359a8df37069 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:48.270114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9e595-d9f1-474e-a766-359a8df37069 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:48.275015 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39722a-8c3b-4b0b-a195-8fe78eca2c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:48.275893 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39722a-8c3b-4b0b-a195-8fe78eca2c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:48.276178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39722a-8c3b-4b0b-a195-8fe78eca2c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:48.276456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39722a-8c3b-4b0b-a195-8fe78eca2c86 admin admin] PATH_INFO: `/v3/projects/6d52008e0a274c3aa3757babb64de9a5/users/c886018fb676434dbce38c25de1b1aa8/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:48.291101 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39722a-8c3b-4b0b-a195-8fe78eca2c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d52008e0a274c3aa3757babb64de9a5, user_id=c886018fb676434dbce38c25de1b1aa8, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:48.293721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39722a-8c3b-4b0b-a195-8fe78eca2c86 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:48.308618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1105/2198] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:33:48 2022] PUT /identity/v3/projects/6d52008e0a274c3aa3757babb64de9a5/users/c886018fb676434dbce38c25de1b1aa8/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:33:48.319556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996911e-b84a-49ac-be8c-41c23dd0fa32 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:48.320048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996911e-b84a-49ac-be8c-41c23dd0fa32 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:48.320145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996911e-b84a-49ac-be8c-41c23dd0fa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:48.348169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0996911e-b84a-49ac-be8c-41c23dd0fa32 None None] MFA Rules not processed for user `c886018fb676434dbce38c25de1b1aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:33:48.349362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0996911e-b84a-49ac-be8c-41c23dd0fa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:48.416776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1094/2199] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:33:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:33:48.427760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39722a-8c3b-4b0b-a195-8fe78eca2c86 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:33:48.429521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9a39722a-8c3b-4b0b-a195-8fe78eca2c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:33:48.492350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39722a-8c3b-4b0b-a195-8fe78eca2c86 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:33:48.496152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ab5fa-c62b-40d3-a487-b41341301156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:33:48.496894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ab5fa-c62b-40d3-a487-b41341301156 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:33:48.497277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ab5fa-c62b-40d3-a487-b41341301156 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:33:48.497622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ab5fa-c62b-40d3-a487-b41341301156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:33:48.508173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ab5fa-c62b-40d3-a487-b41341301156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:33:48.509748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ab5fa-c62b-40d3-a487-b41341301156 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:33:48.522049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1106/2200] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:33:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:33:58.413717 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:33:58.517244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=57,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:34:07.857421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996911e-b84a-49ac-be8c-41c23dd0fa32 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:34:07.921483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996911e-b84a-49ac-be8c-41c23dd0fa32 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:34:07.926039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:34:07.927923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:34:07.928376 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:34:07.928505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] PATH_INFO: `/v3/users/c886018fb676434dbce38c25de1b1aa8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:34:07.929672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c886018fb676434dbce38c25de1b1aa8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:34:07.931623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:34:07.979440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c886018fb676434dbce38c25de1b1aa8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:34:07.980100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c886018fb676434dbce38c25de1b1aa8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:34:07.987198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c886018fb676434dbce38c25de1b1aa8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:34:08.000025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c886018fb676434dbce38c25de1b1aa8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:34:08.000756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c886018fb676434dbce38c25de1b1aa8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:34:08.015352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1095/2201] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:34:07 2022] DELETE /identity/v3/users/c886018fb676434dbce38c25de1b1aa8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:34:08.019915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56196d-6646-4f99-b17a-7db5eb5125a3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:34:08.020562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56196d-6646-4f99-b17a-7db5eb5125a3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:34:08.020562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56196d-6646-4f99-b17a-7db5eb5125a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:34:08.039618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fa56196d-6646-4f99-b17a-7db5eb5125a3 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:34:08.040800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fa56196d-6646-4f99-b17a-7db5eb5125a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:34:08.105504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1107/2202] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:34:08.115689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:34:08.117086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:34:08.179184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4063d5-864b-401d-9660-76f7e7017e10 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:34:08.184691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e9c10-6902-4346-b385-8d70fe82af7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:34:08.185364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e9c10-6902-4346-b385-8d70fe82af7f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:34:08.185780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e9c10-6902-4346-b385-8d70fe82af7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:34:08.186103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e9c10-6902-4346-b385-8d70fe82af7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:34:08.204277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e9c10-6902-4346-b385-8d70fe82af7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:34:08.206212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e9c10-6902-4346-b385-8d70fe82af7f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:34:08.222267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1096/2203] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:34:08.482686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56196d-6646-4f99-b17a-7db5eb5125a3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:34:08.484109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fa56196d-6646-4f99-b17a-7db5eb5125a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:34:08.497456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56196d-6646-4f99-b17a-7db5eb5125a3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:34:08.509865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:34:08.510884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:34:08.511320 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:34:08.511646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] PATH_INFO: `/v3/projects/6d52008e0a274c3aa3757babb64de9a5` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:34:08.513110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d52008e0a274c3aa3757babb64de9a5)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:34:08.515556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:34:08.527393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d66c-4188-46c2-8f71-9422e5de3bc1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:34:08.528042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d66c-4188-46c2-8f71-9422e5de3bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:34:08.528444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d66c-4188-46c2-8f71-9422e5de3bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:34:08.553041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f7c9d66c-4188-46c2-8f71-9422e5de3bc1 None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:34:08.554969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f7c9d66c-4188-46c2-8f71-9422e5de3bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:34:08.578247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:34:08.584034 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d52008e0a274c3aa3757babb64de9a5', 'request_id': 'req-34bcc72b-bcee-48ad-b064-56f3648c1fef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:34:08.593224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d52008e0a274c3aa3757babb64de9a5', 'request_id': 'req-34bcc72b-bcee-48ad-b064-56f3648c1fef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:34:08.601981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d52008e0a274c3aa3757babb64de9a5', 'request_id': 'req-34bcc72b-bcee-48ad-b064-56f3648c1fef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:34:08.614345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] The token cache is being invalidate because project 6d52008e0a274c3aa3757babb64de9a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:34:08.614739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:34:08.615496 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:34:08.618041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1108/2204] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:34:08 2022] DELETE /identity/v3/projects/6d52008e0a274c3aa3757babb64de9a5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:34:08.627505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1097/2205] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jul 21 06:34:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:34:08.641849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:34:08.643807 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:34:08.721969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcc72b-bcee-48ad-b064-56f3648c1fef admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:34:08.728421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e983de-6503-4223-9e28-0ebdd6080594 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:34:08.729190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e983de-6503-4223-9e28-0ebdd6080594 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:34:08.729564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e983de-6503-4223-9e28-0ebdd6080594 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:34:08.729913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e983de-6503-4223-9e28-0ebdd6080594 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:34:08.758421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:34:08.802974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e983de-6503-4223-9e28-0ebdd6080594 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:34:08.804566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e983de-6503-4223-9e28-0ebdd6080594 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:34:08.813506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1109/2206] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:34:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:34:18.619301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=15 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:34:18.809966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=16 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:34:32.496465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4f239-26e2-4749-9082-7019c7635aa2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:34:32.496936 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4f239-26e2-4749-9082-7019c7635aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:34:32.497181 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4f239-26e2-4749-9082-7019c7635aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:34:32.518389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3be4f239-26e2-4749-9082-7019c7635aa2 None None] MFA Rules not processed for user `774dd66c674a4955a439afd30354efab`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:34:32.519624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3be4f239-26e2-4749-9082-7019c7635aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:34:32.579548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1098/2207] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:34:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:34:42.572791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=25 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:35:03.591648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65bdb4-c6b0-4879-ab95-a39336f54470 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:35:03.592197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65bdb4-c6b0-4879-ab95-a39336f54470 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:35:03.592197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65bdb4-c6b0-4879-ab95-a39336f54470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:35:03.609540 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ea65bdb4-c6b0-4879-ab95-a39336f54470 None None] MFA Rules not processed for user `5c54ac39faa84c7db5b9b63360216d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:35:03.610779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ea65bdb4-c6b0-4879-ab95-a39336f54470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:03.666840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1110/2208] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:35:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:35:11.061724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb1d3d-44fc-4da1-acbe-f210396b2367 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:35:11.062202 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb1d3d-44fc-4da1-acbe-f210396b2367 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:35:11.062335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb1d3d-44fc-4da1-acbe-f210396b2367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:35:11.082044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5ceb1d3d-44fc-4da1-acbe-f210396b2367 None None] MFA Rules not processed for user `efafaa14b3e146eba88412c382a8b977`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:35:11.083722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5ceb1d3d-44fc-4da1-acbe-f210396b2367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:11.158821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1099/2209] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:35:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:35:13.383629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65bdb4-c6b0-4879-ab95-a39336f54470 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:35:13.385099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ea65bdb4-c6b0-4879-ab95-a39336f54470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:13.450230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65bdb4-c6b0-4879-ab95-a39336f54470 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:35:13.453894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9b3aa-63d5-4443-8e47-be71cafdf948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:35:13.454578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b3aa-63d5-4443-8e47-be71cafdf948 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:35:13.454869 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b3aa-63d5-4443-8e47-be71cafdf948 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:35:13.455207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b3aa-63d5-4443-8e47-be71cafdf948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:35:13.456927 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a5a9b3aa-63d5-4443-8e47-be71cafdf948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:13.533600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9b3aa-63d5-4443-8e47-be71cafdf948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:35:13.535077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9b3aa-63d5-4443-8e47-be71cafdf948 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:35:13.546559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1111/2210] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:35:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:35:17.469552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb1d3d-44fc-4da1-acbe-f210396b2367 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:35:17.479246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb1d3d-44fc-4da1-acbe-f210396b2367 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:35:17.483483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8b08a-1296-4389-b76c-7ede4a16044c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:35:17.484138 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b08a-1296-4389-b76c-7ede4a16044c service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:35:17.484425 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b08a-1296-4389-b76c-7ede4a16044c service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:35:17.484710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b08a-1296-4389-b76c-7ede4a16044c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:35:17.486625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-34d8b08a-1296-4389-b76c-7ede4a16044c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:17.511192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8b08a-1296-4389-b76c-7ede4a16044c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:35:17.512788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8b08a-1296-4389-b76c-7ede4a16044c service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:35:17.526160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1100/2211] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:35:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:35:21.830989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3965b1-8acc-4293-b9fb-713397eed57c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:35:21.832091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3965b1-8acc-4293-b9fb-713397eed57c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:35:21.832416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3965b1-8acc-4293-b9fb-713397eed57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:35:21.852291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-de3965b1-8acc-4293-b9fb-713397eed57c None None] MFA Rules not processed for user `15ec5e26839f494e98ec70ac49ebe415`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:35:21.853499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-de3965b1-8acc-4293-b9fb-713397eed57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:21.896977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:35:21.912460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1112/2212] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:35:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:35:26.337132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8b08a-1296-4389-b76c-7ede4a16044c service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:35:26.338748 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-34d8b08a-1296-4389-b76c-7ede4a16044c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:26.415750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8b08a-1296-4389-b76c-7ede4a16044c service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:35:26.419996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb756434-29db-47f0-99d3-c895894a5b2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:35:26.420665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb756434-29db-47f0-99d3-c895894a5b2a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:35:26.421006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb756434-29db-47f0-99d3-c895894a5b2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:35:26.421293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb756434-29db-47f0-99d3-c895894a5b2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:35:26.423170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eb756434-29db-47f0-99d3-c895894a5b2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:26.436137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb756434-29db-47f0-99d3-c895894a5b2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:35:26.437546 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb756434-29db-47f0-99d3-c895894a5b2a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:35:26.447082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1101/2213] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:35:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:35:29.559490 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3965b1-8acc-4293-b9fb-713397eed57c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:35:29.560926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-de3965b1-8acc-4293-b9fb-713397eed57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:29.572612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3965b1-8acc-4293-b9fb-713397eed57c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:35:29.576897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc79092-214b-46fc-8143-5d43d38c509d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:35:29.577617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc79092-214b-46fc-8143-5d43d38c509d service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:35:29.577909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc79092-214b-46fc-8143-5d43d38c509d service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:35:29.578286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc79092-214b-46fc-8143-5d43d38c509d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:35:29.580015 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9dc79092-214b-46fc-8143-5d43d38c509d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:29.640998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc79092-214b-46fc-8143-5d43d38c509d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:35:29.642667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc79092-214b-46fc-8143-5d43d38c509d service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:35:29.652994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1113/2214] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:35:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:35:36.443335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=57 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:35:36.724044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb756434-29db-47f0-99d3-c895894a5b2a service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:35:36.725504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eb756434-29db-47f0-99d3-c895894a5b2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:36.741462 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb756434-29db-47f0-99d3-c895894a5b2a service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:35:36.748039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26953e56-e609-4d2f-bfe0-48df084c20b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:35:36.748675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26953e56-e609-4d2f-bfe0-48df084c20b6 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:35:36.748959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26953e56-e609-4d2f-bfe0-48df084c20b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:35:36.749242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26953e56-e609-4d2f-bfe0-48df084c20b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:35:36.751084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-26953e56-e609-4d2f-bfe0-48df084c20b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:36.763040 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26953e56-e609-4d2f-bfe0-48df084c20b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:35:36.764633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26953e56-e609-4d2f-bfe0-48df084c20b6 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:35:36.773876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1102/2215] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:35:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:35:39.647761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=32 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:35:45.727865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2a49b-a048-454a-a954-d242a3dea99a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:35:45.728459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2a49b-a048-454a-a954-d242a3dea99a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:35:45.728459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2a49b-a048-454a-a954-d242a3dea99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:35:45.748420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1ee2a49b-a048-454a-a954-d242a3dea99a None None] MFA Rules not processed for user `8561734cf36e47b5ab2d7242076437ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:35:45.750364 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1ee2a49b-a048-454a-a954-d242a3dea99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:45.758387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1114/2216] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:35:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:35:46.297743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26953e56-e609-4d2f-bfe0-48df084c20b6 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:35:46.306968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26953e56-e609-4d2f-bfe0-48df084c20b6 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:35:46.310747 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a5851-15bc-46e9-a158-89603c995915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:35:46.311386 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a5851-15bc-46e9-a158-89603c995915 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:35:46.311672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a5851-15bc-46e9-a158-89603c995915 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:35:46.311951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a5851-15bc-46e9-a158-89603c995915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:35:46.313731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-598a5851-15bc-46e9-a158-89603c995915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:46.323491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a5851-15bc-46e9-a158-89603c995915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:35:46.325372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a5851-15bc-46e9-a158-89603c995915 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:35:46.333398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1103/2217] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:35:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:35:46.657738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2a49b-a048-454a-a954-d242a3dea99a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:35:46.660742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1ee2a49b-a048-454a-a954-d242a3dea99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:46.746416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2a49b-a048-454a-a954-d242a3dea99a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:35:46.751561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17df241-3d5c-4164-86a5-bfab1e98b57b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da2d85eb18049f196aeefc51d3f1e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:35:46.752218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17df241-3d5c-4164-86a5-bfab1e98b57b service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:35:46.752510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17df241-3d5c-4164-86a5-bfab1e98b57b service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:35:46.752788 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17df241-3d5c-4164-86a5-bfab1e98b57b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:35:46.754589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d17df241-3d5c-4164-86a5-bfab1e98b57b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:35:46.766183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17df241-3d5c-4164-86a5-bfab1e98b57b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:35:46.767789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17df241-3d5c-4164-86a5-bfab1e98b57b service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:35:46.777877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1115/2218] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:35:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:35:56.329756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=12 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:35:56.773397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=35 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:00.875960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a5851-15bc-46e9-a158-89603c995915 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:00.877368 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-598a5851-15bc-46e9-a158-89603c995915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:00.889649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a5851-15bc-46e9-a158-89603c995915 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:00.893465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58b3e8-bf66-488e-8ec3-3d6c38ea7c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:00.894150 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58b3e8-bf66-488e-8ec3-3d6c38ea7c7d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:00.894434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58b3e8-bf66-488e-8ec3-3d6c38ea7c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:00.894814 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58b3e8-bf66-488e-8ec3-3d6c38ea7c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:00.896672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5d58b3e8-bf66-488e-8ec3-3d6c38ea7c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:00.908310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58b3e8-bf66-488e-8ec3-3d6c38ea7c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:00.909785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58b3e8-bf66-488e-8ec3-3d6c38ea7c7d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:00.919286 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1104/2219] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:01.579538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f4829-958d-4ae4-aa1d-7412658930c9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:01.580078 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f4829-958d-4ae4-aa1d-7412658930c9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:01.580448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f4829-958d-4ae4-aa1d-7412658930c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:01.598830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-528f4829-958d-4ae4-aa1d-7412658930c9 None None] MFA Rules not processed for user `5439d1a65154486bb4fdfeeaf8ab4b07`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:01.599925 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-528f4829-958d-4ae4-aa1d-7412658930c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:01.657004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1116/2220] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:02.596501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58b3e8-bf66-488e-8ec3-3d6c38ea7c7d service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:02.598664 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5d58b3e8-bf66-488e-8ec3-3d6c38ea7c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:02.627629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58b3e8-bf66-488e-8ec3-3d6c38ea7c7d service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:02.635734 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f32e1b-161c-4775-80c5-371ca4a4b5f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:02.637712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f32e1b-161c-4775-80c5-371ca4a4b5f1 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:02.638372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f32e1b-161c-4775-80c5-371ca4a4b5f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:02.638863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f32e1b-161c-4775-80c5-371ca4a4b5f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:02.642057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-60f32e1b-161c-4775-80c5-371ca4a4b5f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:02.675257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f32e1b-161c-4775-80c5-371ca4a4b5f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:02.677065 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f32e1b-161c-4775-80c5-371ca4a4b5f1 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:02.690314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1105/2221] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:03.974812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f4829-958d-4ae4-aa1d-7412658930c9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:03.984859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f4829-958d-4ae4-aa1d-7412658930c9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:03.988991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fc343-a0e3-4fb1-b7cb-874d92ca4b05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:03.989639 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fc343-a0e3-4fb1-b7cb-874d92ca4b05 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:03.989926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fc343-a0e3-4fb1-b7cb-874d92ca4b05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:03.990230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fc343-a0e3-4fb1-b7cb-874d92ca4b05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:03.991998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4f3fc343-a0e3-4fb1-b7cb-874d92ca4b05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:04.060214 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fc343-a0e3-4fb1-b7cb-874d92ca4b05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:04.061655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fc343-a0e3-4fb1-b7cb-874d92ca4b05 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:04.072480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1117/2222] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:08.331269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f32e1b-161c-4775-80c5-371ca4a4b5f1 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:08.332759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-60f32e1b-161c-4775-80c5-371ca4a4b5f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:08.359280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f32e1b-161c-4775-80c5-371ca4a4b5f1 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:08.364685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:08.365595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:08.365934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:08.366401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] PATH_INFO: `/v3/users/774dd66c674a4955a439afd30354efab` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:08.367984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=774dd66c674a4955a439afd30354efab)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:08.370134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:08.453326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '774dd66c674a4955a439afd30354efab'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:08.454106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '774dd66c674a4955a439afd30354efab'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:08.463714 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '774dd66c674a4955a439afd30354efab'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:08.481565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '774dd66c674a4955a439afd30354efab'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:08.482900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '774dd66c674a4955a439afd30354efab'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:08.505484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1106/2223] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:08 2022] DELETE /identity/v3/users/774dd66c674a4955a439afd30354efab => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:08.850347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fc343-a0e3-4fb1-b7cb-874d92ca4b05 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:08.867039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fc343-a0e3-4fb1-b7cb-874d92ca4b05 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:08.961353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:08.962274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:08.962557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:08.962835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] PATH_INFO: `/v3/projects/9cd8a1a642a24b519aefdde9ed336536` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:08.964058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cd8a1a642a24b519aefdde9ed336536)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:08.966075 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:08.999060 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:09.001199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:09.050969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:09.072379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cd8a1a642a24b519aefdde9ed336536', 'request_id': 'req-9c0d2568-2de7-430e-b066-d63f629b9221'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:09.084863 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cd8a1a642a24b519aefdde9ed336536', 'request_id': 'req-9c0d2568-2de7-430e-b066-d63f629b9221'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:09.095053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cd8a1a642a24b519aefdde9ed336536', 'request_id': 'req-9c0d2568-2de7-430e-b066-d63f629b9221'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:09.109693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] The token cache is being invalidate because project 9cd8a1a642a24b519aefdde9ed336536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:09.110288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:09.110987 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9c0d2568-2de7-430e-b066-d63f629b9221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:09.116133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1118/2224] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:08 2022] DELETE /identity/v3/projects/9cd8a1a642a24b519aefdde9ed336536 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:09.118194 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70563-bd9b-4986-b359-2b2daa7b2ebd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.118567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70563-bd9b-4986-b359-2b2daa7b2ebd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.118935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70563-bd9b-4986-b359-2b2daa7b2ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.124238 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8afd69-003b-4d3b-89ce-8d80233fc705 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:09.140553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e7b70563-bd9b-4986-b359-2b2daa7b2ebd None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:09.141962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e7b70563-bd9b-4986-b359-2b2daa7b2ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:09.180929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:09.222084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1119/2225] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:36:09.222979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732d75f-bc0c-46d0-97b1-c8307911cfd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:09.224031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732d75f-bc0c-46d0-97b1-c8307911cfd4 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.224031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732d75f-bc0c-46d0-97b1-c8307911cfd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.224263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732d75f-bc0c-46d0-97b1-c8307911cfd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.226255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7732d75f-bc0c-46d0-97b1-c8307911cfd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:09.226845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2edc5-43f8-47c1-9139-601c2a810661 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.227140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2edc5-43f8-47c1-9139-601c2a810661 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.227428 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2edc5-43f8-47c1-9139-601c2a810661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.250090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5ec2edc5-43f8-47c1-9139-601c2a810661 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:09.250316 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732d75f-bc0c-46d0-97b1-c8307911cfd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:09.251685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5ec2edc5-43f8-47c1-9139-601c2a810661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:09.252061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732d75f-bc0c-46d0-97b1-c8307911cfd4 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:09.263094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1120/2226] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:36:09.266958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2edc5-43f8-47c1-9139-601c2a810661 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:09.270434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1107/2227] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:09.279058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2edc5-43f8-47c1-9139-601c2a810661 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:09.283707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47171e9e-aa11-470f-a30f-39db03892c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:09.284376 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47171e9e-aa11-470f-a30f-39db03892c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.284658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47171e9e-aa11-470f-a30f-39db03892c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.284933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47171e9e-aa11-470f-a30f-39db03892c43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.285712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47171e9e-aa11-470f-a30f-39db03892c43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:36:09.286071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47171e9e-aa11-470f-a30f-39db03892c43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:09.287837 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47171e9e-aa11-470f-a30f-39db03892c43 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:09.301767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1121/2228] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:36:09.306851 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732d75f-bc0c-46d0-97b1-c8307911cfd4 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:09.318704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732d75f-bc0c-46d0-97b1-c8307911cfd4 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:09.323492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb396b00-1938-4f7b-b9e4-d45f9a29f18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:09.324234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb396b00-1938-4f7b-b9e4-d45f9a29f18f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.325055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb396b00-1938-4f7b-b9e4-d45f9a29f18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.325334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb396b00-1938-4f7b-b9e4-d45f9a29f18f admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.326075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb396b00-1938-4f7b-b9e4-d45f9a29f18f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:09.327980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb396b00-1938-4f7b-b9e4-d45f9a29f18f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:09.359521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1108/2229] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:09 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:09.365030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47171e9e-aa11-470f-a30f-39db03892c43 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:09.377002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47171e9e-aa11-470f-a30f-39db03892c43 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:09.437130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2cbf9-adcd-48a0-992e-503f633516da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:09.438077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2cbf9-adcd-48a0-992e-503f633516da admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.438367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2cbf9-adcd-48a0-992e-503f633516da admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.438649 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2cbf9-adcd-48a0-992e-503f633516da admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.439354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2cbf9-adcd-48a0-992e-503f633516da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:09.441439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2cbf9-adcd-48a0-992e-503f633516da admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:09.461967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1122/2230] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:09 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:09.466689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb396b00-1938-4f7b-b9e4-d45f9a29f18f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:09.479142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb396b00-1938-4f7b-b9e4-d45f9a29f18f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:09.487045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63297f9f-a6a8-46d8-a876-4404f7d0e04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:09.487739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63297f9f-a6a8-46d8-a876-4404f7d0e04a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.488028 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63297f9f-a6a8-46d8-a876-4404f7d0e04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.488315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63297f9f-a6a8-46d8-a876-4404f7d0e04a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.489066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63297f9f-a6a8-46d8-a876-4404f7d0e04a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:09.490750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63297f9f-a6a8-46d8-a876-4404f7d0e04a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:09.503599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1109/2231] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:09.507653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2cbf9-adcd-48a0-992e-503f633516da admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:09.519224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2cbf9-adcd-48a0-992e-503f633516da admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:09.524445 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500641d-5ec8-48ec-9ff0-9ac0a1bd9aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:09.525197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500641d-5ec8-48ec-9ff0-9ac0a1bd9aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.525563 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500641d-5ec8-48ec-9ff0-9ac0a1bd9aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.526047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500641d-5ec8-48ec-9ff0-9ac0a1bd9aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.526803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500641d-5ec8-48ec-9ff0-9ac0a1bd9aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:09.529131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500641d-5ec8-48ec-9ff0-9ac0a1bd9aa3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:09.544551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1123/2232] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:09.546086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63297f9f-a6a8-46d8-a876-4404f7d0e04a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:09.556438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63297f9f-a6a8-46d8-a876-4404f7d0e04a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:09.561278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f9dae-b1ba-4653-8075-0a590b849786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:09.562215 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f9dae-b1ba-4653-8075-0a590b849786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.562539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f9dae-b1ba-4653-8075-0a590b849786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.562822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f9dae-b1ba-4653-8075-0a590b849786 admin admin] PATH_INFO: `/v3/projects/b05bb5f973c94f79ac00c931fc347539/users/ebe8614a811d43108a7995448e462fa8/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.578665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f9dae-b1ba-4653-8075-0a590b849786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b05bb5f973c94f79ac00c931fc347539, user_id=ebe8614a811d43108a7995448e462fa8, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:09.581389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f9dae-b1ba-4653-8075-0a590b849786 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:09.594684 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1110/2233] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:09 2022] PUT /identity/v3/projects/b05bb5f973c94f79ac00c931fc347539/users/ebe8614a811d43108a7995448e462fa8/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:09.604579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162703fb-46d1-4d3c-8cc0-ac9544c2cd49 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.605283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162703fb-46d1-4d3c-8cc0-ac9544c2cd49 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.606355 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162703fb-46d1-4d3c-8cc0-ac9544c2cd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.634481 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-162703fb-46d1-4d3c-8cc0-ac9544c2cd49 None None] MFA Rules not processed for user `ebe8614a811d43108a7995448e462fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:09.635693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-162703fb-46d1-4d3c-8cc0-ac9544c2cd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:09.654545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:09.708516 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1124/2234] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:09.712876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f9dae-b1ba-4653-8075-0a590b849786 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:09.778900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f9dae-b1ba-4653-8075-0a590b849786 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:09.783598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fba332-23f4-4a3f-bd75-473dbc414214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:09.784257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fba332-23f4-4a3f-bd75-473dbc414214 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.784539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fba332-23f4-4a3f-bd75-473dbc414214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.784817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fba332-23f4-4a3f-bd75-473dbc414214 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.785527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fba332-23f4-4a3f-bd75-473dbc414214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:09.787574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fba332-23f4-4a3f-bd75-473dbc414214 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:09.817804 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1111/2235] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:09 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:09.818906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162703fb-46d1-4d3c-8cc0-ac9544c2cd49 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:09.830200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162703fb-46d1-4d3c-8cc0-ac9544c2cd49 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:09.889168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca35fbc-23d3-472a-a25d-3df0f8cdf979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:09.890154 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35fbc-23d3-472a-a25d-3df0f8cdf979 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.890476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35fbc-23d3-472a-a25d-3df0f8cdf979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.890862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35fbc-23d3-472a-a25d-3df0f8cdf979 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.891757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca35fbc-23d3-472a-a25d-3df0f8cdf979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:09.894044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca35fbc-23d3-472a-a25d-3df0f8cdf979 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:09.920847 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1125/2236] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:09 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:09.924770 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fba332-23f4-4a3f-bd75-473dbc414214 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:09.936057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fba332-23f4-4a3f-bd75-473dbc414214 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:09.940842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b913bf-acb5-40dc-9222-519c1773c38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:09.941388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b913bf-acb5-40dc-9222-519c1773c38a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.941672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b913bf-acb5-40dc-9222-519c1773c38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.941981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b913bf-acb5-40dc-9222-519c1773c38a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.942755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b913bf-acb5-40dc-9222-519c1773c38a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:09.944755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b913bf-acb5-40dc-9222-519c1773c38a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:09.960815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1112/2237] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:36:09.961920 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca35fbc-23d3-472a-a25d-3df0f8cdf979 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:09.973178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca35fbc-23d3-472a-a25d-3df0f8cdf979 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:09.977826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b599c-a02f-4284-8ceb-dde24c3dac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:09.978769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b599c-a02f-4284-8ceb-dde24c3dac39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:09.979048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b599c-a02f-4284-8ceb-dde24c3dac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:09.979326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b599c-a02f-4284-8ceb-dde24c3dac39 admin admin] PATH_INFO: `/v3/projects/b55bc806903a4c60b3045230970381eb/users/091dcd18b873430284eda13792aa51da/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:09.995308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b599c-a02f-4284-8ceb-dde24c3dac39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b55bc806903a4c60b3045230970381eb, user_id=091dcd18b873430284eda13792aa51da, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:09.999266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b599c-a02f-4284-8ceb-dde24c3dac39 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:10.013499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1126/2238] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:09 2022] PUT /identity/v3/projects/b55bc806903a4c60b3045230970381eb/users/091dcd18b873430284eda13792aa51da/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:10.019523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b913bf-acb5-40dc-9222-519c1773c38a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:10.029969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b913bf-acb5-40dc-9222-519c1773c38a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:10.079692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:10.098341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6105d-a8be-44a0-9bf5-aad68b0c8b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:10.099079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6105d-a8be-44a0-9bf5-aad68b0c8b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:10.100171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6105d-a8be-44a0-9bf5-aad68b0c8b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:10.100520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6105d-a8be-44a0-9bf5-aad68b0c8b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:10.101443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6105d-a8be-44a0-9bf5-aad68b0c8b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:10.103073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6105d-a8be-44a0-9bf5-aad68b0c8b37 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:10.117293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1113/2239] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:10.123503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b599c-a02f-4284-8ceb-dde24c3dac39 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:10.140443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b599c-a02f-4284-8ceb-dde24c3dac39 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:10.145494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f835a9-9e0c-4d8f-af8e-614a4b861af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:10.146455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f835a9-9e0c-4d8f-af8e-614a4b861af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:10.146741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f835a9-9e0c-4d8f-af8e-614a4b861af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:10.147021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f835a9-9e0c-4d8f-af8e-614a4b861af2 admin admin] PATH_INFO: `/v3/projects/b55bc806903a4c60b3045230970381eb/users/091dcd18b873430284eda13792aa51da/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:10.149503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f835a9-9e0c-4d8f-af8e-614a4b861af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b55bc806903a4c60b3045230970381eb, user_id=091dcd18b873430284eda13792aa51da, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:10.152018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f835a9-9e0c-4d8f-af8e-614a4b861af2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:10.165029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1127/2240] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:10 2022] PUT /identity/v3/projects/b55bc806903a4c60b3045230970381eb/users/091dcd18b873430284eda13792aa51da/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:36:10.175583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a81383-a76b-452c-94e7-5fd24103372e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:10.175878 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a81383-a76b-452c-94e7-5fd24103372e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:10.176157 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a81383-a76b-452c-94e7-5fd24103372e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:10.212404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e3a81383-a76b-452c-94e7-5fd24103372e None None] MFA Rules not processed for user `091dcd18b873430284eda13792aa51da`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:10.213658 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e3a81383-a76b-452c-94e7-5fd24103372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:10.288993 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f835a9-9e0c-4d8f-af8e-614a4b861af2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:10.291849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-89f835a9-9e0c-4d8f-af8e-614a4b861af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:10.325502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1114/2241] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:10.338141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a81383-a76b-452c-94e7-5fd24103372e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:10.340216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e3a81383-a76b-452c-94e7-5fd24103372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:10.408214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a81383-a76b-452c-94e7-5fd24103372e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:10.412151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f835a9-9e0c-4d8f-af8e-614a4b861af2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:10.412329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b8c02-58ad-4eb5-b69d-5be96fecbcfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:10.412895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b8c02-58ad-4eb5-b69d-5be96fecbcfc service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:10.413182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b8c02-58ad-4eb5-b69d-5be96fecbcfc service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:10.413466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b8c02-58ad-4eb5-b69d-5be96fecbcfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:10.419504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7156a57-e045-46b4-a7fb-c20aa9246d78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:10.420319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7156a57-e045-46b4-a7fb-c20aa9246d78 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:10.420712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7156a57-e045-46b4-a7fb-c20aa9246d78 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:10.421095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7156a57-e045-46b4-a7fb-c20aa9246d78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:10.423687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c7156a57-e045-46b4-a7fb-c20aa9246d78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:10.424769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b8c02-58ad-4eb5-b69d-5be96fecbcfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:10.426378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b8c02-58ad-4eb5-b69d-5be96fecbcfc service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:10.436321 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:10.438184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1115/2242] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:10.439118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b8c02-58ad-4eb5-b69d-5be96fecbcfc service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:10.440842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cb5b8c02-58ad-4eb5-b69d-5be96fecbcfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:10.444100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7156a57-e045-46b4-a7fb-c20aa9246d78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:10.446324 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7156a57-e045-46b4-a7fb-c20aa9246d78 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:10.459168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b8c02-58ad-4eb5-b69d-5be96fecbcfc service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:10.466109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84702b85-3f7e-4bb0-9acc-ae4f11e8259f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:10.466784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84702b85-3f7e-4bb0-9acc-ae4f11e8259f service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:10.467070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84702b85-3f7e-4bb0-9acc-ae4f11e8259f service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:10.467359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84702b85-3f7e-4bb0-9acc-ae4f11e8259f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:10.469630 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-84702b85-3f7e-4bb0-9acc-ae4f11e8259f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:10.473997 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1128/2243] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:36:10.524644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7156a57-e045-46b4-a7fb-c20aa9246d78 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:10.556366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84702b85-3f7e-4bb0-9acc-ae4f11e8259f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:10.559301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84702b85-3f7e-4bb0-9acc-ae4f11e8259f service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:10.570357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1116/2244] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:10.609114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7156a57-e045-46b4-a7fb-c20aa9246d78 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:10.615626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:10.616600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:10.616960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:10.617318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] PATH_INFO: `/v3/users/ebe8614a811d43108a7995448e462fa8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:10.619199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebe8614a811d43108a7995448e462fa8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:10.621895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:10.695495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebe8614a811d43108a7995448e462fa8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:10.716938 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebe8614a811d43108a7995448e462fa8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:10.728551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebe8614a811d43108a7995448e462fa8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:10.729792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebe8614a811d43108a7995448e462fa8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:10.731149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebe8614a811d43108a7995448e462fa8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:10.758851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1129/2245] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:10 2022] DELETE /identity/v3/users/ebe8614a811d43108a7995448e462fa8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:10.764547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617f05-5456-437b-92c9-b4e6ea7056fb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:10.765201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617f05-5456-437b-92c9-b4e6ea7056fb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:10.765602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617f05-5456-437b-92c9-b4e6ea7056fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:10.787045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-26617f05-5456-437b-92c9-b4e6ea7056fb None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:10.788819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-26617f05-5456-437b-92c9-b4e6ea7056fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:10.794980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:10.879888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1117/2246] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:10.890852 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:10.892476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:10.990884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ced5c4-9aac-413a-9f08-17142ae8aca1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:10.998809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0ab14-6071-4e53-9f94-0ec418c1a86d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:10.999484 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0ab14-6071-4e53-9f94-0ec418c1a86d service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:10.999812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0ab14-6071-4e53-9f94-0ec418c1a86d service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:11.000096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0ab14-6071-4e53-9f94-0ec418c1a86d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:11.011354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0ab14-6071-4e53-9f94-0ec418c1a86d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:11.014055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0ab14-6071-4e53-9f94-0ec418c1a86d service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:11.027941 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1130/2247] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:11.317159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26617f05-5456-437b-92c9-b4e6ea7056fb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:11.318794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-26617f05-5456-437b-92c9-b4e6ea7056fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:11.331207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26617f05-5456-437b-92c9-b4e6ea7056fb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:11.335952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:11.336913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:11.337266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:11.337614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] PATH_INFO: `/v3/projects/b05bb5f973c94f79ac00c931fc347539` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:11.338923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b05bb5f973c94f79ac00c931fc347539)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:11.341703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:11.389245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b05bb5f973c94f79ac00c931fc347539', 'request_id': 'req-d26db95b-9146-4549-9e9a-12e72d962b43'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:11.394755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b05bb5f973c94f79ac00c931fc347539', 'request_id': 'req-d26db95b-9146-4549-9e9a-12e72d962b43'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:11.401198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b05bb5f973c94f79ac00c931fc347539', 'request_id': 'req-d26db95b-9146-4549-9e9a-12e72d962b43'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:11.414216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] The token cache is being invalidate because project b05bb5f973c94f79ac00c931fc347539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:11.414825 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:11.415242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:11.416856 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1118/2248] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:11 2022] DELETE /identity/v3/projects/b05bb5f973c94f79ac00c931fc347539 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:11.421796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0ab14-6071-4e53-9f94-0ec418c1a86d service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:11.431994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0ab14-6071-4e53-9f94-0ec418c1a86d service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:11.444890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:11.492617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afb224-5b9f-4ee1-8523-bd4ec2f7bb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:11.493457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afb224-5b9f-4ee1-8523-bd4ec2f7bb8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:11.493757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afb224-5b9f-4ee1-8523-bd4ec2f7bb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:11.494135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afb224-5b9f-4ee1-8523-bd4ec2f7bb8f admin admin] PATH_INFO: `/v3/users/091dcd18b873430284eda13792aa51da` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:11.495468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afb224-5b9f-4ee1-8523-bd4ec2f7bb8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=091dcd18b873430284eda13792aa51da)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:11.499118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afb224-5b9f-4ee1-8523-bd4ec2f7bb8f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:11.546302 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-62afb224-5b9f-4ee1-8523-bd4ec2f7bb8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '091dcd18b873430284eda13792aa51da'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:11.559421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-62afb224-5b9f-4ee1-8523-bd4ec2f7bb8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '091dcd18b873430284eda13792aa51da'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:11.567499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-62afb224-5b9f-4ee1-8523-bd4ec2f7bb8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '091dcd18b873430284eda13792aa51da'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:11.568342 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-62afb224-5b9f-4ee1-8523-bd4ec2f7bb8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '091dcd18b873430284eda13792aa51da'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:11.569123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-62afb224-5b9f-4ee1-8523-bd4ec2f7bb8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '091dcd18b873430284eda13792aa51da'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:11.581966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1131/2249] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:11 2022] DELETE /identity/v3/users/091dcd18b873430284eda13792aa51da => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:11.851748 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:11.853233 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:11.868295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26db95b-9146-4549-9e9a-12e72d962b43 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:11.934467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:11.935350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:11.935777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:11.936058 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] PATH_INFO: `/v3/projects/b55bc806903a4c60b3045230970381eb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:11.937312 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b55bc806903a4c60b3045230970381eb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:11.939380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:11.984841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:12.009882 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b55bc806903a4c60b3045230970381eb', 'request_id': 'req-de281607-3b09-4b5f-951c-b74120d8db01'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:12.018789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b55bc806903a4c60b3045230970381eb', 'request_id': 'req-de281607-3b09-4b5f-951c-b74120d8db01'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:12.026545 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b55bc806903a4c60b3045230970381eb', 'request_id': 'req-de281607-3b09-4b5f-951c-b74120d8db01'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:12.049248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] The token cache is being invalidate because project b55bc806903a4c60b3045230970381eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:12.050060 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:12.050908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de281607-3b09-4b5f-951c-b74120d8db01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:12.056915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1119/2250] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:11 2022] DELETE /identity/v3/projects/b55bc806903a4c60b3045230970381eb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:12.059260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334c1db-15d2-4d29-bb27-4315a1b5971f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.060291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334c1db-15d2-4d29-bb27-4315a1b5971f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.060710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334c1db-15d2-4d29-bb27-4315a1b5971f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.083089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f334c1db-15d2-4d29-bb27-4315a1b5971f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:12.084509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f334c1db-15d2-4d29-bb27-4315a1b5971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:12.168749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1132/2251] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:12.177785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366e055-2853-4ab0-bb9c-7454615e917c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.178126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366e055-2853-4ab0-bb9c-7454615e917c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.178409 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366e055-2853-4ab0-bb9c-7454615e917c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.204697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6366e055-2853-4ab0-bb9c-7454615e917c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:12.206425 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6366e055-2853-4ab0-bb9c-7454615e917c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:12.217985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1120/2252] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:12.221469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334c1db-15d2-4d29-bb27-4315a1b5971f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:12.239178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334c1db-15d2-4d29-bb27-4315a1b5971f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:12.244558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d992e6-85df-4545-8c83-86c9729db971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:12.245246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d992e6-85df-4545-8c83-86c9729db971 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.245530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d992e6-85df-4545-8c83-86c9729db971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.245805 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d992e6-85df-4545-8c83-86c9729db971 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.246832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d992e6-85df-4545-8c83-86c9729db971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:36:12.247176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d992e6-85df-4545-8c83-86c9729db971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:12.248802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d992e6-85df-4545-8c83-86c9729db971 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:12.262909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1133/2253] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:36:12.267480 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366e055-2853-4ab0-bb9c-7454615e917c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:12.277484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366e055-2853-4ab0-bb9c-7454615e917c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:12.282923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47564c-7680-4b69-a7cd-6ae57729e0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:12.283767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47564c-7680-4b69-a7cd-6ae57729e0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.284118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47564c-7680-4b69-a7cd-6ae57729e0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.284467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47564c-7680-4b69-a7cd-6ae57729e0e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.285317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47564c-7680-4b69-a7cd-6ae57729e0e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:12.287883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47564c-7680-4b69-a7cd-6ae57729e0e3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:12.315735 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1121/2254] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:12 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:12.320055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d992e6-85df-4545-8c83-86c9729db971 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:12.330669 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d992e6-85df-4545-8c83-86c9729db971 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:12.352003 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:12.388779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62104eeb-0a53-4d4b-a896-2c6cca310c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:12.389553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62104eeb-0a53-4d4b-a896-2c6cca310c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.389835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62104eeb-0a53-4d4b-a896-2c6cca310c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.390144 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62104eeb-0a53-4d4b-a896-2c6cca310c95 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.390840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62104eeb-0a53-4d4b-a896-2c6cca310c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:12.393443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62104eeb-0a53-4d4b-a896-2c6cca310c95 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:12.415725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1134/2255] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:12 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:12.420204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47564c-7680-4b69-a7cd-6ae57729e0e3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:12.430643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47564c-7680-4b69-a7cd-6ae57729e0e3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:12.435629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594f9af-3015-4cd9-ade3-45aa7f9aae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:12.436283 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f9af-3015-4cd9-ade3-45aa7f9aae25 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.436566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f9af-3015-4cd9-ade3-45aa7f9aae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.436842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f9af-3015-4cd9-ade3-45aa7f9aae25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.437572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594f9af-3015-4cd9-ade3-45aa7f9aae25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:12.439199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594f9af-3015-4cd9-ade3-45aa7f9aae25 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:12.449978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1122/2256] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:12.454429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62104eeb-0a53-4d4b-a896-2c6cca310c95 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:12.464244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62104eeb-0a53-4d4b-a896-2c6cca310c95 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:12.469737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc975834-c5fc-4e4f-a9af-6c5a87729558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:12.470418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc975834-c5fc-4e4f-a9af-6c5a87729558 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.470728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc975834-c5fc-4e4f-a9af-6c5a87729558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.471005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc975834-c5fc-4e4f-a9af-6c5a87729558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.471727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc975834-c5fc-4e4f-a9af-6c5a87729558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:12.473291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc975834-c5fc-4e4f-a9af-6c5a87729558 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:12.483649 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1135/2257] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:12.488025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594f9af-3015-4cd9-ade3-45aa7f9aae25 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:12.498528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594f9af-3015-4cd9-ade3-45aa7f9aae25 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:12.503508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18283c-905d-43a8-9569-be45748df7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:12.504403 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18283c-905d-43a8-9569-be45748df7df admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.504691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18283c-905d-43a8-9569-be45748df7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.504976 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18283c-905d-43a8-9569-be45748df7df admin admin] PATH_INFO: `/v3/projects/f61f39db2a3847c0b79ec158eed6364b/users/dd2c2ec8de8f464db0a6ace7d366511e/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.522990 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18283c-905d-43a8-9569-be45748df7df admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61f39db2a3847c0b79ec158eed6364b, user_id=dd2c2ec8de8f464db0a6ace7d366511e, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:12.526048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18283c-905d-43a8-9569-be45748df7df admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:12.540812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1123/2258] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:12 2022] PUT /identity/v3/projects/f61f39db2a3847c0b79ec158eed6364b/users/dd2c2ec8de8f464db0a6ace7d366511e/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:12.551742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b595c-6ca7-45a4-a6cc-03ed3301e0e8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.552087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b595c-6ca7-45a4-a6cc-03ed3301e0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.552449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b595c-6ca7-45a4-a6cc-03ed3301e0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.594859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d03b595c-6ca7-45a4-a6cc-03ed3301e0e8 None None] MFA Rules not processed for user `dd2c2ec8de8f464db0a6ace7d366511e`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:12.596106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d03b595c-6ca7-45a4-a6cc-03ed3301e0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:12.677171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1136/2259] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:12.681851 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18283c-905d-43a8-9569-be45748df7df admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:12.769616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18283c-905d-43a8-9569-be45748df7df admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:12.777169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236204bf-57e0-4787-9330-982865a363a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:12.777882 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236204bf-57e0-4787-9330-982865a363a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.778277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236204bf-57e0-4787-9330-982865a363a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.778563 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236204bf-57e0-4787-9330-982865a363a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.779300 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236204bf-57e0-4787-9330-982865a363a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:12.781147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236204bf-57e0-4787-9330-982865a363a5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:12.809978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1124/2260] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:12 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:12.815530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b595c-6ca7-45a4-a6cc-03ed3301e0e8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:12.827776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b595c-6ca7-45a4-a6cc-03ed3301e0e8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:12.888799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552e559-4363-4673-b405-7cf1d2e54324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:12.889491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552e559-4363-4673-b405-7cf1d2e54324 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.889781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552e559-4363-4673-b405-7cf1d2e54324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.890094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552e559-4363-4673-b405-7cf1d2e54324 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.890824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552e559-4363-4673-b405-7cf1d2e54324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:12.892741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552e559-4363-4673-b405-7cf1d2e54324 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:12.919259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1137/2261] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:12 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:36:12.921501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236204bf-57e0-4787-9330-982865a363a5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:12.937776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236204bf-57e0-4787-9330-982865a363a5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:12.943194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b74ac3-dacf-4dfb-b3fc-540fae05bd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:12.943964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b74ac3-dacf-4dfb-b3fc-540fae05bd0d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.944251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b74ac3-dacf-4dfb-b3fc-540fae05bd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.944533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b74ac3-dacf-4dfb-b3fc-540fae05bd0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.945279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b74ac3-dacf-4dfb-b3fc-540fae05bd0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:12.947518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b74ac3-dacf-4dfb-b3fc-540fae05bd0d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:12.961707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1125/2262] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:12.966061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552e559-4363-4673-b405-7cf1d2e54324 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:12.976247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552e559-4363-4673-b405-7cf1d2e54324 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:12.981808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb00a8-4f23-412e-a494-74a2830197b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:12.983226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb00a8-4f23-412e-a494-74a2830197b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:12.983430 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb00a8-4f23-412e-a494-74a2830197b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:12.983716 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb00a8-4f23-412e-a494-74a2830197b5 admin admin] PATH_INFO: `/v3/projects/2881e8d5f0d54a2a9273a9e8d80ce505/users/e1ee2dd9b68d41a48775cf332550aa34/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:12.999763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb00a8-4f23-412e-a494-74a2830197b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2881e8d5f0d54a2a9273a9e8d80ce505, user_id=e1ee2dd9b68d41a48775cf332550aa34, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:13.003022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb00a8-4f23-412e-a494-74a2830197b5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:13.011128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:13.018445 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1138/2263] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:12 2022] PUT /identity/v3/projects/2881e8d5f0d54a2a9273a9e8d80ce505/users/e1ee2dd9b68d41a48775cf332550aa34/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:36:13.022564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b74ac3-dacf-4dfb-b3fc-540fae05bd0d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:13.033610 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b74ac3-dacf-4dfb-b3fc-540fae05bd0d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:13.078721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:13.110670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a3c83-7a10-48e4-9c49-7ab4e0b8f3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:13.111387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a3c83-7a10-48e4-9c49-7ab4e0b8f3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:13.111672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a3c83-7a10-48e4-9c49-7ab4e0b8f3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:13.111950 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a3c83-7a10-48e4-9c49-7ab4e0b8f3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:13.112691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a3c83-7a10-48e4-9c49-7ab4e0b8f3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:13.114347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a3c83-7a10-48e4-9c49-7ab4e0b8f3ca admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:13.125874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1126/2264] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:13.131601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb00a8-4f23-412e-a494-74a2830197b5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:13.143125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb00a8-4f23-412e-a494-74a2830197b5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:13.148337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffd1e7-98c5-4054-be2d-d8f2000cda04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:13.149499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffd1e7-98c5-4054-be2d-d8f2000cda04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:13.149851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffd1e7-98c5-4054-be2d-d8f2000cda04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:13.150239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffd1e7-98c5-4054-be2d-d8f2000cda04 admin admin] PATH_INFO: `/v3/projects/2881e8d5f0d54a2a9273a9e8d80ce505/users/e1ee2dd9b68d41a48775cf332550aa34/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:13.152331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffd1e7-98c5-4054-be2d-d8f2000cda04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2881e8d5f0d54a2a9273a9e8d80ce505, user_id=e1ee2dd9b68d41a48775cf332550aa34, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:13.155323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffd1e7-98c5-4054-be2d-d8f2000cda04 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:13.166653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1139/2265] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:13 2022] PUT /identity/v3/projects/2881e8d5f0d54a2a9273a9e8d80ce505/users/e1ee2dd9b68d41a48775cf332550aa34/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:13.178068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8e6e-29b9-4bfa-8569-7288a0800786 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:13.178068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8e6e-29b9-4bfa-8569-7288a0800786 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:13.178068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8e6e-29b9-4bfa-8569-7288a0800786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:13.210778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-938c8e6e-29b9-4bfa-8569-7288a0800786 None None] MFA Rules not processed for user `e1ee2dd9b68d41a48775cf332550aa34`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:13.212039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-938c8e6e-29b9-4bfa-8569-7288a0800786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:13.313436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1127/2266] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:13.329793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffd1e7-98c5-4054-be2d-d8f2000cda04 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:13.331580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-58ffd1e7-98c5-4054-be2d-d8f2000cda04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:13.406473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffd1e7-98c5-4054-be2d-d8f2000cda04 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:13.410419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86b8ca-51fa-406a-b052-1b8a61637e34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:13.411113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86b8ca-51fa-406a-b052-1b8a61637e34 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:13.411402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86b8ca-51fa-406a-b052-1b8a61637e34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:13.411686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86b8ca-51fa-406a-b052-1b8a61637e34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:13.421403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86b8ca-51fa-406a-b052-1b8a61637e34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:13.422897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86b8ca-51fa-406a-b052-1b8a61637e34 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:13.436378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1140/2267] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:36:13.550361 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c8e6e-29b9-4bfa-8569-7288a0800786 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:13.563520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c8e6e-29b9-4bfa-8569-7288a0800786 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:13.567198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45bd72-537d-4474-aa19-7797fb216009 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:13.567862 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45bd72-537d-4474-aa19-7797fb216009 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:13.568148 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45bd72-537d-4474-aa19-7797fb216009 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:13.568432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45bd72-537d-4474-aa19-7797fb216009 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:13.627810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45bd72-537d-4474-aa19-7797fb216009 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:13.629270 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45bd72-537d-4474-aa19-7797fb216009 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:13.638768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1128/2268] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:16.251165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86b8ca-51fa-406a-b052-1b8a61637e34 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:16.317778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86b8ca-51fa-406a-b052-1b8a61637e34 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:16.322354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:16.323231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:16.323512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:16.323789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] PATH_INFO: `/v3/users/dd2c2ec8de8f464db0a6ace7d366511e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:16.326140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2c2ec8de8f464db0a6ace7d366511e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:16.328106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:16.375291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2c2ec8de8f464db0a6ace7d366511e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:16.386893 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2c2ec8de8f464db0a6ace7d366511e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:16.393613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2c2ec8de8f464db0a6ace7d366511e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:16.394292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2c2ec8de8f464db0a6ace7d366511e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:16.394902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2c2ec8de8f464db0a6ace7d366511e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:16.409375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1141/2269] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:16 2022] DELETE /identity/v3/users/dd2c2ec8de8f464db0a6ace7d366511e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:16.413906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521276b-51c9-46ce-9c6c-0331486dbb9b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:16.414384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521276b-51c9-46ce-9c6c-0331486dbb9b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:16.414481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521276b-51c9-46ce-9c6c-0331486dbb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:16.431854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c521276b-51c9-46ce-9c6c-0331486dbb9b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:16.433039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c521276b-51c9-46ce-9c6c-0331486dbb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:16.479025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:16.493219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1129/2270] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:16.503004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:16.504448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:16.514769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:16.564043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47eb394-ad42-44fa-9b8c-e957ae60c7a3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:16.567645 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec24209-9703-4b88-a67b-99880faffd65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:16.568297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec24209-9703-4b88-a67b-99880faffd65 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:16.568584 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec24209-9703-4b88-a67b-99880faffd65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:16.568861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec24209-9703-4b88-a67b-99880faffd65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:16.577812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec24209-9703-4b88-a67b-99880faffd65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:16.579291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec24209-9703-4b88-a67b-99880faffd65 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:16.588810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1142/2271] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:16.945860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521276b-51c9-46ce-9c6c-0331486dbb9b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:16.947441 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c521276b-51c9-46ce-9c6c-0331486dbb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:16.962204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521276b-51c9-46ce-9c6c-0331486dbb9b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:16.967942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:16.968858 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:16.969110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:16.969417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] PATH_INFO: `/v3/projects/f61f39db2a3847c0b79ec158eed6364b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:16.970694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f61f39db2a3847c0b79ec158eed6364b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:16.972923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:17.032189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f61f39db2a3847c0b79ec158eed6364b', 'request_id': 'req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.037828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f61f39db2a3847c0b79ec158eed6364b', 'request_id': 'req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.044732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f61f39db2a3847c0b79ec158eed6364b', 'request_id': 'req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.060785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] The token cache is being invalidate because project f61f39db2a3847c0b79ec158eed6364b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:17.061149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.061855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.063504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1130/2272] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:16 2022] DELETE /identity/v3/projects/f61f39db2a3847c0b79ec158eed6364b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:17.067815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec24209-9703-4b88-a67b-99880faffd65 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:17.081002 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec24209-9703-4b88-a67b-99880faffd65 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:17.143331 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c769aed-6baa-4af8-9319-f7b943ab60e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:17.144207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c769aed-6baa-4af8-9319-f7b943ab60e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:17.144488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c769aed-6baa-4af8-9319-f7b943ab60e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:17.144764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c769aed-6baa-4af8-9319-f7b943ab60e7 admin admin] PATH_INFO: `/v3/users/e1ee2dd9b68d41a48775cf332550aa34` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:17.145931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c769aed-6baa-4af8-9319-f7b943ab60e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ee2dd9b68d41a48775cf332550aa34)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:17.147883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c769aed-6baa-4af8-9319-f7b943ab60e7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:17.193865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c769aed-6baa-4af8-9319-f7b943ab60e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ee2dd9b68d41a48775cf332550aa34'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.206356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c769aed-6baa-4af8-9319-f7b943ab60e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ee2dd9b68d41a48775cf332550aa34'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.213207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c769aed-6baa-4af8-9319-f7b943ab60e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ee2dd9b68d41a48775cf332550aa34'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.213882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c769aed-6baa-4af8-9319-f7b943ab60e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ee2dd9b68d41a48775cf332550aa34'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.214527 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c769aed-6baa-4af8-9319-f7b943ab60e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ee2dd9b68d41a48775cf332550aa34'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.227059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1143/2273] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:17 2022] DELETE /identity/v3/users/e1ee2dd9b68d41a48775cf332550aa34 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:17.498239 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:17.499726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:17.516167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c675134-385b-4b2c-b3d1-bedcc25d4ae7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:17.576327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:17.577129 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:17.577418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:17.577700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] PATH_INFO: `/v3/projects/2881e8d5f0d54a2a9273a9e8d80ce505` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:17.578927 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2881e8d5f0d54a2a9273a9e8d80ce505)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:17.580847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:17.627513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2881e8d5f0d54a2a9273a9e8d80ce505', 'request_id': 'req-9e579dcb-da2a-40a1-bd6c-7edf0d247589'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.633085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2881e8d5f0d54a2a9273a9e8d80ce505', 'request_id': 'req-9e579dcb-da2a-40a1-bd6c-7edf0d247589'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.639310 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2881e8d5f0d54a2a9273a9e8d80ce505', 'request_id': 'req-9e579dcb-da2a-40a1-bd6c-7edf0d247589'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.653034 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] The token cache is being invalidate because project 2881e8d5f0d54a2a9273a9e8d80ce505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:17.653321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.653970 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e579dcb-da2a-40a1-bd6c-7edf0d247589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:17.655520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1131/2274] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:17 2022] DELETE /identity/v3/projects/2881e8d5f0d54a2a9273a9e8d80ce505 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:17.660427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea201f62-5f86-4d45-94a1-793e5367da66 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:17.660891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea201f62-5f86-4d45-94a1-793e5367da66 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:17.660987 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea201f62-5f86-4d45-94a1-793e5367da66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:17.679877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ea201f62-5f86-4d45-94a1-793e5367da66 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:17.681077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ea201f62-5f86-4d45-94a1-793e5367da66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:17.706756 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:17.742882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1144/2275] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:36:17.752148 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8a37c-05a7-4a3f-81ef-4dd344875c02 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:17.752507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8a37c-05a7-4a3f-81ef-4dd344875c02 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:17.752902 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8a37c-05a7-4a3f-81ef-4dd344875c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:17.765356 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:17.771970 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-34d8a37c-05a7-4a3f-81ef-4dd344875c02 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:17.773660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-34d8a37c-05a7-4a3f-81ef-4dd344875c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:17.782893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1132/2276] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:17.787070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea201f62-5f86-4d45-94a1-793e5367da66 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:17.797502 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea201f62-5f86-4d45-94a1-793e5367da66 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:17.802214 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61249d-5f2b-4595-9543-9f5ea061221e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:17.802867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61249d-5f2b-4595-9543-9f5ea061221e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:17.803180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61249d-5f2b-4595-9543-9f5ea061221e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:17.803456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61249d-5f2b-4595-9543-9f5ea061221e admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:17.804188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61249d-5f2b-4595-9543-9f5ea061221e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:36:17.804493 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61249d-5f2b-4595-9543-9f5ea061221e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:17.806136 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61249d-5f2b-4595-9543-9f5ea061221e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:17.817964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1145/2277] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:36:17.822574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8a37c-05a7-4a3f-81ef-4dd344875c02 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:17.831915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8a37c-05a7-4a3f-81ef-4dd344875c02 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:17.836444 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bdef4-3434-41e2-8e55-5be8428b05c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:17.837091 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bdef4-3434-41e2-8e55-5be8428b05c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:17.837371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bdef4-3434-41e2-8e55-5be8428b05c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:17.837646 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bdef4-3434-41e2-8e55-5be8428b05c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:17.838380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bdef4-3434-41e2-8e55-5be8428b05c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:17.840864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bdef4-3434-41e2-8e55-5be8428b05c8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:17.866889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1133/2278] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:17 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:17.871894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61249d-5f2b-4595-9543-9f5ea061221e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:17.882009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61249d-5f2b-4595-9543-9f5ea061221e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:17.944634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf410fe-4582-4faa-b4cc-50f13522e1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:17.945320 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf410fe-4582-4faa-b4cc-50f13522e1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:17.945633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf410fe-4582-4faa-b4cc-50f13522e1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:17.945908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf410fe-4582-4faa-b4cc-50f13522e1a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:17.946643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf410fe-4582-4faa-b4cc-50f13522e1a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:17.948577 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf410fe-4582-4faa-b4cc-50f13522e1a5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:17.973654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1146/2279] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:17 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:36:17.974858 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bdef4-3434-41e2-8e55-5be8428b05c8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:17.985622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bdef4-3434-41e2-8e55-5be8428b05c8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:17.990548 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b14eb-dd01-43ea-85f1-32e53a4b8d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:17.991467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b14eb-dd01-43ea-85f1-32e53a4b8d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:17.991753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b14eb-dd01-43ea-85f1-32e53a4b8d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:17.992048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b14eb-dd01-43ea-85f1-32e53a4b8d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:17.992787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b14eb-dd01-43ea-85f1-32e53a4b8d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:17.994425 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b14eb-dd01-43ea-85f1-32e53a4b8d24 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.008920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1134/2280] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:18.010242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf410fe-4582-4faa-b4cc-50f13522e1a5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:18.021099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf410fe-4582-4faa-b4cc-50f13522e1a5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:18.026022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dbb0d-f090-4d99-8fa8-781355d66983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:18.026683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dbb0d-f090-4d99-8fa8-781355d66983 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.026965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dbb0d-f090-4d99-8fa8-781355d66983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.027512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dbb0d-f090-4d99-8fa8-781355d66983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.028261 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dbb0d-f090-4d99-8fa8-781355d66983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:18.029830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dbb0d-f090-4d99-8fa8-781355d66983 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.040522 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1147/2281] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:18.045010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b14eb-dd01-43ea-85f1-32e53a4b8d24 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:18.060447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b14eb-dd01-43ea-85f1-32e53a4b8d24 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:18.065125 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fb375-f204-46d5-ad1b-55a18e3cdffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:18.066080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fb375-f204-46d5-ad1b-55a18e3cdffd admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.066381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fb375-f204-46d5-ad1b-55a18e3cdffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.066653 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fb375-f204-46d5-ad1b-55a18e3cdffd admin admin] PATH_INFO: `/v3/projects/86bfef3bffcb4d6090da416637884599/users/c673bafbce974731a34be7d7785bb136/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.082666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fb375-f204-46d5-ad1b-55a18e3cdffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bfef3bffcb4d6090da416637884599, user_id=c673bafbce974731a34be7d7785bb136, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:18.085225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fb375-f204-46d5-ad1b-55a18e3cdffd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.098135 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1135/2282] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:18 2022] PUT /identity/v3/projects/86bfef3bffcb4d6090da416637884599/users/c673bafbce974731a34be7d7785bb136/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:18.109267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6646b-7bdd-4000-b2e1-33bfd340f362 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.109560 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6646b-7bdd-4000-b2e1-33bfd340f362 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.109839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6646b-7bdd-4000-b2e1-33bfd340f362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.139471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-55f6646b-7bdd-4000-b2e1-33bfd340f362 None None] MFA Rules not processed for user `c673bafbce974731a34be7d7785bb136`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:18.140864 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-55f6646b-7bdd-4000-b2e1-33bfd340f362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:18.221261 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1148/2283] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:18.225659 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fb375-f204-46d5-ad1b-55a18e3cdffd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:18.310562 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fb375-f204-46d5-ad1b-55a18e3cdffd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:18.316571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade48e3-a7a8-4b34-b047-98e0415dff3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:18.316853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade48e3-a7a8-4b34-b047-98e0415dff3d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.317136 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade48e3-a7a8-4b34-b047-98e0415dff3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.317729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade48e3-a7a8-4b34-b047-98e0415dff3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.318631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade48e3-a7a8-4b34-b047-98e0415dff3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:18.320939 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade48e3-a7a8-4b34-b047-98e0415dff3d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.353989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1136/2284] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:18 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:36:18.358356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6646b-7bdd-4000-b2e1-33bfd340f362 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:18.372383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6646b-7bdd-4000-b2e1-33bfd340f362 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:18.412712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:18.454949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed202df6-c5dd-425e-aad0-0edcbf47a653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:18.455725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed202df6-c5dd-425e-aad0-0edcbf47a653 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.456083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed202df6-c5dd-425e-aad0-0edcbf47a653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.456431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed202df6-c5dd-425e-aad0-0edcbf47a653 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.457207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed202df6-c5dd-425e-aad0-0edcbf47a653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:18.459240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed202df6-c5dd-425e-aad0-0edcbf47a653 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.481463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1149/2285] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:18 2022] POST /identity/v3/users => generated 300 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:18.486015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade48e3-a7a8-4b34-b047-98e0415dff3d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:18.497920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade48e3-a7a8-4b34-b047-98e0415dff3d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:18.503384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae918808-e290-4826-b27e-e5cbf4483510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:18.504019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae918808-e290-4826-b27e-e5cbf4483510 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.504019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae918808-e290-4826-b27e-e5cbf4483510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.504281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae918808-e290-4826-b27e-e5cbf4483510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.505114 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae918808-e290-4826-b27e-e5cbf4483510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:18.507009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae918808-e290-4826-b27e-e5cbf4483510 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.519247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1137/2286] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:36:18.523685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed202df6-c5dd-425e-aad0-0edcbf47a653 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:18.538278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed202df6-c5dd-425e-aad0-0edcbf47a653 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:18.545969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6d3d6-0a59-4b43-b516-36d7fd4f3e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:18.548761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6d3d6-0a59-4b43-b516-36d7fd4f3e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.549261 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6d3d6-0a59-4b43-b516-36d7fd4f3e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.549869 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6d3d6-0a59-4b43-b516-36d7fd4f3e41 admin admin] PATH_INFO: `/v3/projects/724b83577ef9445c9a1af0dcbf44f194/users/00045b5521ce4b3da9c9fd20900f2995/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.568883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6d3d6-0a59-4b43-b516-36d7fd4f3e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=724b83577ef9445c9a1af0dcbf44f194, user_id=00045b5521ce4b3da9c9fd20900f2995, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:18.571565 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6d3d6-0a59-4b43-b516-36d7fd4f3e41 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.588989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1150/2287] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:18 2022] PUT /identity/v3/projects/724b83577ef9445c9a1af0dcbf44f194/users/00045b5521ce4b3da9c9fd20900f2995/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:18.591588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae918808-e290-4826-b27e-e5cbf4483510 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:18.601496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae918808-e290-4826-b27e-e5cbf4483510 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:18.663220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8711ff-4581-48ff-b5ce-8d490a312750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:18.663898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8711ff-4581-48ff-b5ce-8d490a312750 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.664221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8711ff-4581-48ff-b5ce-8d490a312750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.664505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8711ff-4581-48ff-b5ce-8d490a312750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.665241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8711ff-4581-48ff-b5ce-8d490a312750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:18.666933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8711ff-4581-48ff-b5ce-8d490a312750 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.677521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1138/2288] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:18.681638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6d3d6-0a59-4b43-b516-36d7fd4f3e41 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:18.692270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6d3d6-0a59-4b43-b516-36d7fd4f3e41 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:18.696883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70287b9-72ef-4d03-96ad-b42590d2448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:18.697776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70287b9-72ef-4d03-96ad-b42590d2448b admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.698100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70287b9-72ef-4d03-96ad-b42590d2448b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.698382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70287b9-72ef-4d03-96ad-b42590d2448b admin admin] PATH_INFO: `/v3/projects/724b83577ef9445c9a1af0dcbf44f194/users/00045b5521ce4b3da9c9fd20900f2995/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.700631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70287b9-72ef-4d03-96ad-b42590d2448b admin admin] RBAC: Authorizing `identity:create_grant(project_id=724b83577ef9445c9a1af0dcbf44f194, user_id=00045b5521ce4b3da9c9fd20900f2995, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:18.703318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70287b9-72ef-4d03-96ad-b42590d2448b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.714074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1151/2289] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:18 2022] PUT /identity/v3/projects/724b83577ef9445c9a1af0dcbf44f194/users/00045b5521ce4b3da9c9fd20900f2995/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:18.725810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71114cf7-d8c4-44af-a023-656cd938b419 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.726133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71114cf7-d8c4-44af-a023-656cd938b419 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.726457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71114cf7-d8c4-44af-a023-656cd938b419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.752339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:18.755619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-71114cf7-d8c4-44af-a023-656cd938b419 None None] MFA Rules not processed for user `00045b5521ce4b3da9c9fd20900f2995`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:18.757515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-71114cf7-d8c4-44af-a023-656cd938b419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:18.770266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03159b0-200d-4b78-90ae-d6195e141688 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.770568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03159b0-200d-4b78-90ae-d6195e141688 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.770855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03159b0-200d-4b78-90ae-d6195e141688 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.772422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1152/2290] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 06:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 06:36:18.783014 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db35c-ee81-4338-bd79-b58f39c634cb None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.783441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db35c-ee81-4338-bd79-b58f39c634cb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.783795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db35c-ee81-4338-bd79-b58f39c634cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.805718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4e5db35c-ee81-4338-bd79-b58f39c634cb None None] MFA Rules not processed for user `1da2d85eb18049f196aeefc51d3f1e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:18.807014 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4e5db35c-ee81-4338-bd79-b58f39c634cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:18.837988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1139/2291] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:18.854128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71114cf7-d8c4-44af-a023-656cd938b419 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:18.858260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-71114cf7-d8c4-44af-a023-656cd938b419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:18.888532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1153/2292] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 06:36:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:36:18.889390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5db35c-ee81-4338-bd79-b58f39c634cb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:18.890835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4e5db35c-ee81-4338-bd79-b58f39c634cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:18.924612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71114cf7-d8c4-44af-a023-656cd938b419 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:18.928362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5a798-804b-4553-94df-cb360548775e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:18.929121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a798-804b-4553-94df-cb360548775e service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.929413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a798-804b-4553-94df-cb360548775e service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.929692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a798-804b-4553-94df-cb360548775e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.940331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5a798-804b-4553-94df-cb360548775e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:18.941767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5a798-804b-4553-94df-cb360548775e service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.952679 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1140/2293] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:18.953599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5a798-804b-4553-94df-cb360548775e service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:18.954908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9de5a798-804b-4553-94df-cb360548775e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:18.964288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5db35c-ee81-4338-bd79-b58f39c634cb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:18.970117 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5a798-804b-4553-94df-cb360548775e service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:18.971027 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:18.972060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.972446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.972942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] PATH_INFO: `/v3/users/efafaa14b3e146eba88412c382a8b977` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.974555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efafaa14b3e146eba88412c382a8b977)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:18.975503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c934b-63a6-4aa9-a6c6-121a6db5e66f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da2d85eb18049f196aeefc51d3f1e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:18.976159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c934b-63a6-4aa9-a6c6-121a6db5e66f service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:18.976442 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c934b-63a6-4aa9-a6c6-121a6db5e66f service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:18.976722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c934b-63a6-4aa9-a6c6-121a6db5e66f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:18.977201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.988623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c934b-63a6-4aa9-a6c6-121a6db5e66f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:18.990517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c934b-63a6-4aa9-a6c6-121a6db5e66f service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:18.997885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:19.005326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1141/2294] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:19.076104 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efafaa14b3e146eba88412c382a8b977'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.092629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efafaa14b3e146eba88412c382a8b977'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.100249 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efafaa14b3e146eba88412c382a8b977'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.101973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efafaa14b3e146eba88412c382a8b977'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.101973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efafaa14b3e146eba88412c382a8b977'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.125777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1154/2295] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:18 2022] DELETE /identity/v3/users/efafaa14b3e146eba88412c382a8b977 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:19.143199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c934b-63a6-4aa9-a6c6-121a6db5e66f service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:19.158323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c934b-63a6-4aa9-a6c6-121a6db5e66f service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:19.225059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c295c-b807-45a2-b190-d4aeab0c4ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:19.225959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c295c-b807-45a2-b190-d4aeab0c4ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:19.226263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c295c-b807-45a2-b190-d4aeab0c4ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:19.226541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c295c-b807-45a2-b190-d4aeab0c4ad2 admin admin] PATH_INFO: `/v3/users/c673bafbce974731a34be7d7785bb136` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:19.228004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c295c-b807-45a2-b190-d4aeab0c4ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c673bafbce974731a34be7d7785bb136)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:19.229920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c295c-b807-45a2-b190-d4aeab0c4ad2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:19.266940 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:19.282421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c05c295c-b807-45a2-b190-d4aeab0c4ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c673bafbce974731a34be7d7785bb136'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.283243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c05c295c-b807-45a2-b190-d4aeab0c4ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c673bafbce974731a34be7d7785bb136'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.290180 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c05c295c-b807-45a2-b190-d4aeab0c4ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c673bafbce974731a34be7d7785bb136'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.305108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c05c295c-b807-45a2-b190-d4aeab0c4ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c673bafbce974731a34be7d7785bb136'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.305833 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c05c295c-b807-45a2-b190-d4aeab0c4ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c673bafbce974731a34be7d7785bb136'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.319404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:19.320350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1142/2296] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:19 2022] DELETE /identity/v3/users/c673bafbce974731a34be7d7785bb136 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:19.321368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:19.325481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c9719-3135-4fcb-950f-0513007c994d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:19.325786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c9719-3135-4fcb-950f-0513007c994d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:19.326104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c9719-3135-4fcb-950f-0513007c994d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:19.338558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c214d-8511-441d-b52b-8eaddde5e780 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:19.343894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7c1c9719-3135-4fcb-950f-0513007c994d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:19.345041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7c1c9719-3135-4fcb-950f-0513007c994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:19.419859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1143/2297] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:19.432314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c9719-3135-4fcb-950f-0513007c994d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:19.433831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7c1c9719-3135-4fcb-950f-0513007c994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:19.445138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:19.446060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:19.446350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:19.446627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] PATH_INFO: `/v3/projects/854daeeee974410c9a0afafee795e72c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:19.447959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=854daeeee974410c9a0afafee795e72c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:19.450224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:19.497117 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c9719-3135-4fcb-950f-0513007c994d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:19.501494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854daeeee974410c9a0afafee795e72c', 'request_id': 'req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.508958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854daeeee974410c9a0afafee795e72c', 'request_id': 'req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.516247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854daeeee974410c9a0afafee795e72c', 'request_id': 'req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.527279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] The token cache is being invalidate because project 854daeeee974410c9a0afafee795e72c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:19.527605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.528270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:19.529975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1155/2298] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:19 2022] DELETE /identity/v3/projects/854daeeee974410c9a0afafee795e72c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:36:19.552588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846120a4-c89f-4375-a2a5-270a92529764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:19.553268 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846120a4-c89f-4375-a2a5-270a92529764 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:19.553559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846120a4-c89f-4375-a2a5-270a92529764 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:19.553836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846120a4-c89f-4375-a2a5-270a92529764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:19.555789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-846120a4-c89f-4375-a2a5-270a92529764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:19.578002 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:19.630526 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846120a4-c89f-4375-a2a5-270a92529764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:19.631897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846120a4-c89f-4375-a2a5-270a92529764 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:19.644105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1144/2299] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:19.923152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:19.924697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:19.937780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac48c99-3d67-434b-8fa1-3ae82dcd4d5d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:19.942823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:19.944206 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:19.944499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:19.944982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] PATH_INFO: `/v3/projects/86bfef3bffcb4d6090da416637884599` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:19.946471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86bfef3bffcb4d6090da416637884599)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:19.948353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:19.966279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:19.996919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86bfef3bffcb4d6090da416637884599', 'request_id': 'req-d0035893-8e92-4408-bdee-258485d87c61'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.004141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86bfef3bffcb4d6090da416637884599', 'request_id': 'req-d0035893-8e92-4408-bdee-258485d87c61'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.010336 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86bfef3bffcb4d6090da416637884599', 'request_id': 'req-d0035893-8e92-4408-bdee-258485d87c61'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.021386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] The token cache is being invalidate because project 86bfef3bffcb4d6090da416637884599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:20.021674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.022334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.024353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1156/2300] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:19 2022] DELETE /identity/v3/projects/86bfef3bffcb4d6090da416637884599 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:20.028879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846120a4-c89f-4375-a2a5-270a92529764 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:20.038888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846120a4-c89f-4375-a2a5-270a92529764 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:20.095796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e2a64-2f0b-4b0f-a12e-7b7cb0ee5fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:20.096673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e2a64-2f0b-4b0f-a12e-7b7cb0ee5fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:20.096958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e2a64-2f0b-4b0f-a12e-7b7cb0ee5fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:20.097260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e2a64-2f0b-4b0f-a12e-7b7cb0ee5fab admin admin] PATH_INFO: `/v3/users/00045b5521ce4b3da9c9fd20900f2995` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:20.098484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e2a64-2f0b-4b0f-a12e-7b7cb0ee5fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=00045b5521ce4b3da9c9fd20900f2995)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:20.100464 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e2a64-2f0b-4b0f-a12e-7b7cb0ee5fab admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:20.145015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-649e2a64-2f0b-4b0f-a12e-7b7cb0ee5fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00045b5521ce4b3da9c9fd20900f2995'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.145765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-649e2a64-2f0b-4b0f-a12e-7b7cb0ee5fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00045b5521ce4b3da9c9fd20900f2995'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.153154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-649e2a64-2f0b-4b0f-a12e-7b7cb0ee5fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00045b5521ce4b3da9c9fd20900f2995'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.166962 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-649e2a64-2f0b-4b0f-a12e-7b7cb0ee5fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00045b5521ce4b3da9c9fd20900f2995'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.167602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-649e2a64-2f0b-4b0f-a12e-7b7cb0ee5fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00045b5521ce4b3da9c9fd20900f2995'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.186005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1145/2301] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:20 2022] DELETE /identity/v3/users/00045b5521ce4b3da9c9fd20900f2995 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:20.513545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:20.515264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:20.530146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0035893-8e92-4408-bdee-258485d87c61 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:20.602429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:20.603317 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:20.603602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:20.603878 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] PATH_INFO: `/v3/projects/724b83577ef9445c9a1af0dcbf44f194` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:20.605118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] RBAC: Authorizing `identity:delete_project(project_id=724b83577ef9445c9a1af0dcbf44f194)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:20.607961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:20.657929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '724b83577ef9445c9a1af0dcbf44f194', 'request_id': 'req-1203c74f-4a2c-4395-b3f4-5024320d592d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.665128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '724b83577ef9445c9a1af0dcbf44f194', 'request_id': 'req-1203c74f-4a2c-4395-b3f4-5024320d592d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.677672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '724b83577ef9445c9a1af0dcbf44f194', 'request_id': 'req-1203c74f-4a2c-4395-b3f4-5024320d592d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.691961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] The token cache is being invalidate because project 724b83577ef9445c9a1af0dcbf44f194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:20.692260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.692923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1203c74f-4a2c-4395-b3f4-5024320d592d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:20.694520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1157/2302] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:20 2022] DELETE /identity/v3/projects/724b83577ef9445c9a1af0dcbf44f194 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:20.699462 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12129673-8c89-46f5-9aeb-96a6d8841519 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:20.700093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12129673-8c89-46f5-9aeb-96a6d8841519 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:20.700093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12129673-8c89-46f5-9aeb-96a6d8841519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:20.718615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-12129673-8c89-46f5-9aeb-96a6d8841519 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:20.719892 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-12129673-8c89-46f5-9aeb-96a6d8841519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:20.753233 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:20.785546 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1146/2303] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:20.794000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20830075-6462-4ebe-b52f-b33bb1ffd63b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:20.794305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20830075-6462-4ebe-b52f-b33bb1ffd63b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:20.794592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20830075-6462-4ebe-b52f-b33bb1ffd63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:20.812853 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-20830075-6462-4ebe-b52f-b33bb1ffd63b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:20.814088 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-20830075-6462-4ebe-b52f-b33bb1ffd63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:20.821911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1158/2304] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:20.826187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12129673-8c89-46f5-9aeb-96a6d8841519 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:20.835869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12129673-8c89-46f5-9aeb-96a6d8841519 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:20.840997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf78cda-15d4-44b1-b3a8-055480ea3b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:20.841893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78cda-15d4-44b1-b3a8-055480ea3b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:20.842029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78cda-15d4-44b1-b3a8-055480ea3b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:20.842239 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78cda-15d4-44b1-b3a8-055480ea3b8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:20.842976 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf78cda-15d4-44b1-b3a8-055480ea3b8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:36:20.843284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf78cda-15d4-44b1-b3a8-055480ea3b8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:20.844928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf78cda-15d4-44b1-b3a8-055480ea3b8d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:20.857400 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1147/2305] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:36:20.861660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20830075-6462-4ebe-b52f-b33bb1ffd63b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:20.871534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20830075-6462-4ebe-b52f-b33bb1ffd63b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:20.876401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef0b7e-d3b9-43b1-a3d7-07d1e7cf51cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:20.877059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef0b7e-d3b9-43b1-a3d7-07d1e7cf51cb admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:20.877345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef0b7e-d3b9-43b1-a3d7-07d1e7cf51cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:20.877625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef0b7e-d3b9-43b1-a3d7-07d1e7cf51cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:20.878378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef0b7e-d3b9-43b1-a3d7-07d1e7cf51cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:20.880277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef0b7e-d3b9-43b1-a3d7-07d1e7cf51cb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:20.910823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1159/2306] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:20 2022] POST /identity/v3/projects => generated 407 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:36:20.915319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf78cda-15d4-44b1-b3a8-055480ea3b8d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:20.925975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf78cda-15d4-44b1-b3a8-055480ea3b8d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:20.985918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879c4a2-e799-4c15-ad51-c2ddb358097c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:20.986899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879c4a2-e799-4c15-ad51-c2ddb358097c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:20.987343 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879c4a2-e799-4c15-ad51-c2ddb358097c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:20.987828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879c4a2-e799-4c15-ad51-c2ddb358097c admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:20.988540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879c4a2-e799-4c15-ad51-c2ddb358097c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:20.990496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879c4a2-e799-4c15-ad51-c2ddb358097c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:21.009820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1148/2307] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:20 2022] POST /identity/v3/users => generated 304 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:21.014200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef0b7e-d3b9-43b1-a3d7-07d1e7cf51cb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:21.024922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef0b7e-d3b9-43b1-a3d7-07d1e7cf51cb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:21.030675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03d779-0703-4165-8830-7ab688dcf6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:21.031365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03d779-0703-4165-8830-7ab688dcf6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.031653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03d779-0703-4165-8830-7ab688dcf6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.031933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03d779-0703-4165-8830-7ab688dcf6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.032705 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03d779-0703-4165-8830-7ab688dcf6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:21.034529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03d779-0703-4165-8830-7ab688dcf6c6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:21.035844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:21.048510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1160/2308] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:21.052515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879c4a2-e799-4c15-ad51-c2ddb358097c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:21.062987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879c4a2-e799-4c15-ad51-c2ddb358097c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:21.067673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392d185-43db-4588-a9ad-131bbda5af45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:21.068393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392d185-43db-4588-a9ad-131bbda5af45 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.068679 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392d185-43db-4588-a9ad-131bbda5af45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.068960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392d185-43db-4588-a9ad-131bbda5af45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.069690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392d185-43db-4588-a9ad-131bbda5af45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:21.071329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392d185-43db-4588-a9ad-131bbda5af45 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:21.081898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1149/2309] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:21.086144 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03d779-0703-4165-8830-7ab688dcf6c6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:21.096631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03d779-0703-4165-8830-7ab688dcf6c6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:21.101200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d66c9-f014-4953-af76-6a6261d235d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:21.102110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d66c9-f014-4953-af76-6a6261d235d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.102394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d66c9-f014-4953-af76-6a6261d235d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.102670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d66c9-f014-4953-af76-6a6261d235d2 admin admin] PATH_INFO: `/v3/projects/18dd9b29fb704c559fc8a5f3e29e9ac1/users/47a801e6784e4dd6a0ebe4030d97905e/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.117411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d66c9-f014-4953-af76-6a6261d235d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18dd9b29fb704c559fc8a5f3e29e9ac1, user_id=47a801e6784e4dd6a0ebe4030d97905e, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:21.120071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d66c9-f014-4953-af76-6a6261d235d2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:21.133038 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1161/2310] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:21 2022] PUT /identity/v3/projects/18dd9b29fb704c559fc8a5f3e29e9ac1/users/47a801e6784e4dd6a0ebe4030d97905e/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:21.143403 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde667a-4c2a-4873-9bba-9b367208072f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.143697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde667a-4c2a-4873-9bba-9b367208072f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.144010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde667a-4c2a-4873-9bba-9b367208072f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.172642 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fcde667a-4c2a-4873-9bba-9b367208072f None None] MFA Rules not processed for user `47a801e6784e4dd6a0ebe4030d97905e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:21.173864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fcde667a-4c2a-4873-9bba-9b367208072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:21.238863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1150/2311] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:21 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:21.243480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d66c9-f014-4953-af76-6a6261d235d2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:21.311353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d66c9-f014-4953-af76-6a6261d235d2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:21.316414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a1dd-8b65-4d72-b212-f32821052421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:21.316926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a1dd-8b65-4d72-b212-f32821052421 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.317148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a1dd-8b65-4d72-b212-f32821052421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.317426 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a1dd-8b65-4d72-b212-f32821052421 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.318289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932a1dd-8b65-4d72-b212-f32821052421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:21.320423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932a1dd-8b65-4d72-b212-f32821052421 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:21.348976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1162/2312] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:21 2022] POST /identity/v3/projects => generated 407 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:21.353269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde667a-4c2a-4873-9bba-9b367208072f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:21.367979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde667a-4c2a-4873-9bba-9b367208072f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:21.398359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:21.437973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ed4f6-ed8f-414a-b691-80398cb56a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:21.438660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ed4f6-ed8f-414a-b691-80398cb56a1b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.438942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ed4f6-ed8f-414a-b691-80398cb56a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.439220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ed4f6-ed8f-414a-b691-80398cb56a1b admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.439923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ed4f6-ed8f-414a-b691-80398cb56a1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:21.441877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ed4f6-ed8f-414a-b691-80398cb56a1b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:21.469039 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1151/2313] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:21 2022] POST /identity/v3/users => generated 310 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:21.469880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a1dd-8b65-4d72-b212-f32821052421 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:21.482043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a1dd-8b65-4d72-b212-f32821052421 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:21.488081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5c67f-4f32-4c42-8ec2-07306ba96f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:21.489612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5c67f-4f32-4c42-8ec2-07306ba96f7e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.489890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5c67f-4f32-4c42-8ec2-07306ba96f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.490290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5c67f-4f32-4c42-8ec2-07306ba96f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.491142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5c67f-4f32-4c42-8ec2-07306ba96f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:21.492801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5c67f-4f32-4c42-8ec2-07306ba96f7e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:21.511645 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1163/2314] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:21.511793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ed4f6-ed8f-414a-b691-80398cb56a1b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:21.522925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ed4f6-ed8f-414a-b691-80398cb56a1b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:21.529827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37aa7d1-e4b4-46c1-b86b-e031fd8ac044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:21.530766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37aa7d1-e4b4-46c1-b86b-e031fd8ac044 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.531048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37aa7d1-e4b4-46c1-b86b-e031fd8ac044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.531323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37aa7d1-e4b4-46c1-b86b-e031fd8ac044 admin admin] PATH_INFO: `/v3/projects/556590962f4c4097bdef27c50ce66359/users/1d4ef188e7974dd59aef9781346afdbb/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.548999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37aa7d1-e4b4-46c1-b86b-e031fd8ac044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556590962f4c4097bdef27c50ce66359, user_id=1d4ef188e7974dd59aef9781346afdbb, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:21.551701 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37aa7d1-e4b4-46c1-b86b-e031fd8ac044 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:21.564953 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1152/2315] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:21 2022] PUT /identity/v3/projects/556590962f4c4097bdef27c50ce66359/users/1d4ef188e7974dd59aef9781346afdbb/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:36:21.568945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5c67f-4f32-4c42-8ec2-07306ba96f7e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:21.582574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5c67f-4f32-4c42-8ec2-07306ba96f7e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:21.643997 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ca000-1860-4d0b-afec-28c88d882c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:21.644678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ca000-1860-4d0b-afec-28c88d882c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.644966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ca000-1860-4d0b-afec-28c88d882c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.645246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ca000-1860-4d0b-afec-28c88d882c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.646216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ca000-1860-4d0b-afec-28c88d882c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:21.647801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ca000-1860-4d0b-afec-28c88d882c16 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:21.658343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1164/2316] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:36:21.662591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37aa7d1-e4b4-46c1-b86b-e031fd8ac044 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:21.673352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37aa7d1-e4b4-46c1-b86b-e031fd8ac044 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:21.679293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344671e8-ee3e-4348-becb-d9dbf5e68d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:21.680417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344671e8-ee3e-4348-becb-d9dbf5e68d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.680590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344671e8-ee3e-4348-becb-d9dbf5e68d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.680987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344671e8-ee3e-4348-becb-d9dbf5e68d39 admin admin] PATH_INFO: `/v3/projects/556590962f4c4097bdef27c50ce66359/users/1d4ef188e7974dd59aef9781346afdbb/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.683664 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344671e8-ee3e-4348-becb-d9dbf5e68d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556590962f4c4097bdef27c50ce66359, user_id=1d4ef188e7974dd59aef9781346afdbb, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:21.687822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344671e8-ee3e-4348-becb-d9dbf5e68d39 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:21.698775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1153/2317] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:21 2022] PUT /identity/v3/projects/556590962f4c4097bdef27c50ce66359/users/1d4ef188e7974dd59aef9781346afdbb/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:21.710555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dab95-9c13-4d3c-b31e-9b30b9b28dfd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.710852 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dab95-9c13-4d3c-b31e-9b30b9b28dfd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.711181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dab95-9c13-4d3c-b31e-9b30b9b28dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.739535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e14dab95-9c13-4d3c-b31e-9b30b9b28dfd None None] MFA Rules not processed for user `1d4ef188e7974dd59aef9781346afdbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:21.740817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e14dab95-9c13-4d3c-b31e-9b30b9b28dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:21.783808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:21.830322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1165/2318] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:21 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:21.841735 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344671e8-ee3e-4348-becb-d9dbf5e68d39 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:21.843522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-344671e8-ee3e-4348-becb-d9dbf5e68d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:21.904755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344671e8-ee3e-4348-becb-d9dbf5e68d39 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:21.908732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3c997-56c9-47a4-9b10-64f7f0b99425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:21.909388 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3c997-56c9-47a4-9b10-64f7f0b99425 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:21.909675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3c997-56c9-47a4-9b10-64f7f0b99425 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:21.909978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3c997-56c9-47a4-9b10-64f7f0b99425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:21.919707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3c997-56c9-47a4-9b10-64f7f0b99425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:21.921138 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3c997-56c9-47a4-9b10-64f7f0b99425 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:21.931147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1154/2319] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:21 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:22.085861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dab95-9c13-4d3c-b31e-9b30b9b28dfd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:22.096420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dab95-9c13-4d3c-b31e-9b30b9b28dfd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:22.100348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc892db2-34f6-4ba1-bbfe-b06431d3ba83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:22.101074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc892db2-34f6-4ba1-bbfe-b06431d3ba83 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:22.101437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc892db2-34f6-4ba1-bbfe-b06431d3ba83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:22.101787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc892db2-34f6-4ba1-bbfe-b06431d3ba83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:22.162927 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc892db2-34f6-4ba1-bbfe-b06431d3ba83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:22.164526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc892db2-34f6-4ba1-bbfe-b06431d3ba83 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:22.176463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1166/2320] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:22 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:30.667146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3c997-56c9-47a4-9b10-64f7f0b99425 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:30.737394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3c997-56c9-47a4-9b10-64f7f0b99425 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:30.742621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:30.743833 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:30.744114 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:30.744397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] PATH_INFO: `/v3/users/47a801e6784e4dd6a0ebe4030d97905e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:30.745623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a801e6784e4dd6a0ebe4030d97905e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:30.748017 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:30.776192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:30.799023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a801e6784e4dd6a0ebe4030d97905e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:30.799700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a801e6784e4dd6a0ebe4030d97905e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:30.806942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a801e6784e4dd6a0ebe4030d97905e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:30.820712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a801e6784e4dd6a0ebe4030d97905e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:30.821417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a801e6784e4dd6a0ebe4030d97905e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:30.836959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1155/2321] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:30 2022] DELETE /identity/v3/users/47a801e6784e4dd6a0ebe4030d97905e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:30.842022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd94d-06fe-4172-b36f-3a3c9007bc03 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:30.842521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd94d-06fe-4172-b36f-3a3c9007bc03 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:30.842620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd94d-06fe-4172-b36f-3a3c9007bc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:30.861976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-432fd94d-06fe-4172-b36f-3a3c9007bc03 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:30.863360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-432fd94d-06fe-4172-b36f-3a3c9007bc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:30.940304 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1167/2322] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:30.951111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:30.952554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:31.021773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cdf8a-dbac-4f39-a0e4-39a1a4e30fbb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:31.025903 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36f24e-873e-4aa4-972b-d4e6bf849fb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:31.026584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36f24e-873e-4aa4-972b-d4e6bf849fb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:31.026871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36f24e-873e-4aa4-972b-d4e6bf849fb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:31.027148 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36f24e-873e-4aa4-972b-d4e6bf849fb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:31.036713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36f24e-873e-4aa4-972b-d4e6bf849fb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:31.038280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36f24e-873e-4aa4-972b-d4e6bf849fb3 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:31.049536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1156/2323] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:31.437615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fd94d-06fe-4172-b36f-3a3c9007bc03 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:31.439084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-432fd94d-06fe-4172-b36f-3a3c9007bc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:31.451227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fd94d-06fe-4172-b36f-3a3c9007bc03 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:31.455787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:31.456652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:31.456945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:31.457227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] PATH_INFO: `/v3/projects/18dd9b29fb704c559fc8a5f3e29e9ac1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:31.458552 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=18dd9b29fb704c559fc8a5f3e29e9ac1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:31.460531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:31.521673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18dd9b29fb704c559fc8a5f3e29e9ac1', 'request_id': 'req-34ffb09b-c838-41ad-bb29-4b647232cb1c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:31.530671 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18dd9b29fb704c559fc8a5f3e29e9ac1', 'request_id': 'req-34ffb09b-c838-41ad-bb29-4b647232cb1c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:31.537999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18dd9b29fb704c559fc8a5f3e29e9ac1', 'request_id': 'req-34ffb09b-c838-41ad-bb29-4b647232cb1c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:31.539208 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:31.551940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] The token cache is being invalidate because project 18dd9b29fb704c559fc8a5f3e29e9ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:31.552254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:31.553209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:31.554888 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1168/2324] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:31 2022] DELETE /identity/v3/projects/18dd9b29fb704c559fc8a5f3e29e9ac1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:31.558986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36f24e-873e-4aa4-972b-d4e6bf849fb3 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:31.570102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36f24e-873e-4aa4-972b-d4e6bf849fb3 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:31.630631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e773f5-b9b1-478c-9788-4c840e446445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:31.631540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e773f5-b9b1-478c-9788-4c840e446445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:31.631818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e773f5-b9b1-478c-9788-4c840e446445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:31.632087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e773f5-b9b1-478c-9788-4c840e446445 admin admin] PATH_INFO: `/v3/users/1d4ef188e7974dd59aef9781346afdbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:31.633279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e773f5-b9b1-478c-9788-4c840e446445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d4ef188e7974dd59aef9781346afdbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:31.635224 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e773f5-b9b1-478c-9788-4c840e446445 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:31.686997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-26e773f5-b9b1-478c-9788-4c840e446445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d4ef188e7974dd59aef9781346afdbb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:31.687671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-26e773f5-b9b1-478c-9788-4c840e446445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d4ef188e7974dd59aef9781346afdbb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:31.688897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:31.696298 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-26e773f5-b9b1-478c-9788-4c840e446445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d4ef188e7974dd59aef9781346afdbb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:31.713620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-26e773f5-b9b1-478c-9788-4c840e446445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d4ef188e7974dd59aef9781346afdbb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:31.714466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-26e773f5-b9b1-478c-9788-4c840e446445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d4ef188e7974dd59aef9781346afdbb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:31.729033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1157/2325] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:31 2022] DELETE /identity/v3/users/1d4ef188e7974dd59aef9781346afdbb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:32.007616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:32.009169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:32.030127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffb09b-c838-41ad-bb29-4b647232cb1c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:32.102238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:32.103032 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.103319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.103599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] PATH_INFO: `/v3/projects/556590962f4c4097bdef27c50ce66359` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.105279 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=556590962f4c4097bdef27c50ce66359)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:32.107564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:32.146245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050a199-8fde-42cb-9dfa-03813f816e1d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.146518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050a199-8fde-42cb-9dfa-03813f816e1d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.146946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050a199-8fde-42cb-9dfa-03813f816e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.166518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9050a199-8fde-42cb-9dfa-03813f816e1d None None] MFA Rules not processed for user `5c54ac39faa84c7db5b9b63360216d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:32.167725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9050a199-8fde-42cb-9dfa-03813f816e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:32.169137 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '556590962f4c4097bdef27c50ce66359', 'request_id': 'req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:32.178393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '556590962f4c4097bdef27c50ce66359', 'request_id': 'req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:32.187102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '556590962f4c4097bdef27c50ce66359', 'request_id': 'req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:32.198771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] The token cache is being invalidate because project 556590962f4c4097bdef27c50ce66359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:32.199439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:32.200907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d4b4cbb-fc1b-4a97-80b1-7f16e9a94b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:32.203998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1169/2326] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:32 2022] DELETE /identity/v3/projects/556590962f4c4097bdef27c50ce66359 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:32.208531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3ff66-3af7-4b10-b02f-f315036d9182 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.208826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3ff66-3af7-4b10-b02f-f315036d9182 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.209111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3ff66-3af7-4b10-b02f-f315036d9182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.231958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b3b3ff66-3af7-4b10-b02f-f315036d9182 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:32.232649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1158/2327] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:32.233474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b3b3ff66-3af7-4b10-b02f-f315036d9182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:32.307554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1170/2328] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:36:32.315522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93c297-b370-482f-8b77-c275c7b52267 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.315819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93c297-b370-482f-8b77-c275c7b52267 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.316122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93c297-b370-482f-8b77-c275c7b52267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.336005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-db93c297-b370-482f-8b77-c275c7b52267 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:32.337204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-db93c297-b370-482f-8b77-c275c7b52267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:32.346644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1159/2329] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:32.350871 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3ff66-3af7-4b10-b02f-f315036d9182 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:32.362039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3ff66-3af7-4b10-b02f-f315036d9182 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:32.366542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1f0df-8814-4794-8586-546b1a201955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:32.367203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1f0df-8814-4794-8586-546b1a201955 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.367485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1f0df-8814-4794-8586-546b1a201955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.367764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1f0df-8814-4794-8586-546b1a201955 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.368640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1f0df-8814-4794-8586-546b1a201955 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:36:32.369057 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1f0df-8814-4794-8586-546b1a201955 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:32.370686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1f0df-8814-4794-8586-546b1a201955 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:32.386065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1171/2330] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:36:32.387127 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93c297-b370-482f-8b77-c275c7b52267 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:32.397375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93c297-b370-482f-8b77-c275c7b52267 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:32.402265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea76bf4-4432-4272-aa15-048be9c748e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:32.403025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea76bf4-4432-4272-aa15-048be9c748e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.404267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea76bf4-4432-4272-aa15-048be9c748e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.404544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea76bf4-4432-4272-aa15-048be9c748e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.405241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea76bf4-4432-4272-aa15-048be9c748e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:32.407248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea76bf4-4432-4272-aa15-048be9c748e7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:32.434617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1160/2331] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:32 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:36:32.439190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1f0df-8814-4794-8586-546b1a201955 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:32.449176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1f0df-8814-4794-8586-546b1a201955 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:32.453469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:32.510579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dda22-1e96-4baa-868d-8b0f9dc065be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:32.511343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dda22-1e96-4baa-868d-8b0f9dc065be admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.511695 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dda22-1e96-4baa-868d-8b0f9dc065be admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.512060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dda22-1e96-4baa-868d-8b0f9dc065be admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.513169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538dda22-1e96-4baa-868d-8b0f9dc065be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:32.515119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538dda22-1e96-4baa-868d-8b0f9dc065be admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:32.539174 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1172/2332] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:32 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:32.540514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea76bf4-4432-4272-aa15-048be9c748e7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:32.543022 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dda22-1e96-4baa-868d-8b0f9dc065be admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:32.545410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-538dda22-1e96-4baa-868d-8b0f9dc065be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:32.551750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea76bf4-4432-4272-aa15-048be9c748e7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:32.556865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db5229-8af9-42c1-805b-f0eb61c399cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:32.557613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db5229-8af9-42c1-805b-f0eb61c399cb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.557900 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db5229-8af9-42c1-805b-f0eb61c399cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.558224 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db5229-8af9-42c1-805b-f0eb61c399cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.558959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db5229-8af9-42c1-805b-f0eb61c399cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:32.561266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db5229-8af9-42c1-805b-f0eb61c399cb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:32.574571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1161/2333] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:32.575587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db5229-8af9-42c1-805b-f0eb61c399cb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:32.585293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db5229-8af9-42c1-805b-f0eb61c399cb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:32.589995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d252f54-5000-45d8-ad36-c52d0c3741e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:32.590784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d252f54-5000-45d8-ad36-c52d0c3741e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.591078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d252f54-5000-45d8-ad36-c52d0c3741e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.591352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d252f54-5000-45d8-ad36-c52d0c3741e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.592080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d252f54-5000-45d8-ad36-c52d0c3741e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:32.593669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d252f54-5000-45d8-ad36-c52d0c3741e3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:32.608219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1162/2334] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:36:32.608385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dda22-1e96-4baa-868d-8b0f9dc065be admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:32.609190 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d252f54-5000-45d8-ad36-c52d0c3741e3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:32.610607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf971d04-1ce4-4ddd-93e6-d8ccef934afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:32.611257 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971d04-1ce4-4ddd-93e6-d8ccef934afe service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.611541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971d04-1ce4-4ddd-93e6-d8ccef934afe service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.611817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971d04-1ce4-4ddd-93e6-d8ccef934afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.613602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cf971d04-1ce4-4ddd-93e6-d8ccef934afe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:32.618909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d252f54-5000-45d8-ad36-c52d0c3741e3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:32.624699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64832f3-c694-4044-bc70-f314cc4e3779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:32.624851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf971d04-1ce4-4ddd-93e6-d8ccef934afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:32.625575 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64832f3-c694-4044-bc70-f314cc4e3779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.625859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64832f3-c694-4044-bc70-f314cc4e3779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.626172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64832f3-c694-4044-bc70-f314cc4e3779 admin admin] PATH_INFO: `/v3/projects/c41aa54869d842f5b17a34e0d99364dc/users/e572cce6a688456cbcde3149a107c301/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.626315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf971d04-1ce4-4ddd-93e6-d8ccef934afe service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:32.634651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1173/2335] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:36:32.641381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64832f3-c694-4044-bc70-f314cc4e3779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41aa54869d842f5b17a34e0d99364dc, user_id=e572cce6a688456cbcde3149a107c301, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:32.644058 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64832f3-c694-4044-bc70-f314cc4e3779 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:32.657189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1163/2336] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:32 2022] PUT /identity/v3/projects/c41aa54869d842f5b17a34e0d99364dc/users/e572cce6a688456cbcde3149a107c301/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:36:32.667409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b55752-7a26-400c-8c58-dd21b7f82b77 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.667710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b55752-7a26-400c-8c58-dd21b7f82b77 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.667990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b55752-7a26-400c-8c58-dd21b7f82b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.695193 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-83b55752-7a26-400c-8c58-dd21b7f82b77 None None] MFA Rules not processed for user `e572cce6a688456cbcde3149a107c301`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:32.696420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-83b55752-7a26-400c-8c58-dd21b7f82b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:32.763659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1174/2337] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:32.774859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64832f3-c694-4044-bc70-f314cc4e3779 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:32.776634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e64832f3-c694-4044-bc70-f314cc4e3779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:32.837315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64832f3-c694-4044-bc70-f314cc4e3779 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:32.841630 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c2b2b-bea5-4d01-ac54-226967d9e8bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:32.842412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c2b2b-bea5-4d01-ac54-226967d9e8bf service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:32.842771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c2b2b-bea5-4d01-ac54-226967d9e8bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:32.843112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c2b2b-bea5-4d01-ac54-226967d9e8bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:32.853205 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c2b2b-bea5-4d01-ac54-226967d9e8bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:32.854950 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c2b2b-bea5-4d01-ac54-226967d9e8bf service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:32.859637 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:32.867591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1164/2338] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:40.422089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b55752-7a26-400c-8c58-dd21b7f82b77 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:40.451760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:40.485252 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b55752-7a26-400c-8c58-dd21b7f82b77 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:40.489798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:40.490706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:40.490993 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:40.491276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] PATH_INFO: `/v3/users/e572cce6a688456cbcde3149a107c301` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:40.492463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e572cce6a688456cbcde3149a107c301)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:40.494411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:40.537453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e572cce6a688456cbcde3149a107c301'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:40.549390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e572cce6a688456cbcde3149a107c301'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:40.556270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e572cce6a688456cbcde3149a107c301'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:40.556939 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e572cce6a688456cbcde3149a107c301'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:40.557549 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e572cce6a688456cbcde3149a107c301'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:40.570689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1175/2339] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:40 2022] DELETE /identity/v3/users/e572cce6a688456cbcde3149a107c301 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:40.575551 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3264ce2-6738-4a28-a11a-95f7d0b1f5aa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:40.576040 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3264ce2-6738-4a28-a11a-95f7d0b1f5aa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:40.576154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3264ce2-6738-4a28-a11a-95f7d0b1f5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:40.593455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b3264ce2-6738-4a28-a11a-95f7d0b1f5aa None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:40.594688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b3264ce2-6738-4a28-a11a-95f7d0b1f5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:40.658910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1165/2340] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:40.668616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:40.670897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:40.729568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa64b9-f598-40a4-9bd2-805330671eae admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:40.733954 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8318334-094b-49ed-b24c-73a7a10c5a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:40.734625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8318334-094b-49ed-b24c-73a7a10c5a1d service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:40.734914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8318334-094b-49ed-b24c-73a7a10c5a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:40.735198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8318334-094b-49ed-b24c-73a7a10c5a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:40.744697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8318334-094b-49ed-b24c-73a7a10c5a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:40.746216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8318334-094b-49ed-b24c-73a7a10c5a1d service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:40.756882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1176/2341] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:41.022508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3264ce2-6738-4a28-a11a-95f7d0b1f5aa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:41.023977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b3264ce2-6738-4a28-a11a-95f7d0b1f5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:41.042403 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3264ce2-6738-4a28-a11a-95f7d0b1f5aa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:41.047971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:41.048842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:41.049120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:41.049395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] PATH_INFO: `/v3/projects/c41aa54869d842f5b17a34e0d99364dc` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:41.050624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c41aa54869d842f5b17a34e0d99364dc)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:41.052553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:41.113794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c41aa54869d842f5b17a34e0d99364dc', 'request_id': 'req-803d109d-dcbf-4d88-9308-1d974a7cb7e3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:41.119665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c41aa54869d842f5b17a34e0d99364dc', 'request_id': 'req-803d109d-dcbf-4d88-9308-1d974a7cb7e3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:41.126391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c41aa54869d842f5b17a34e0d99364dc', 'request_id': 'req-803d109d-dcbf-4d88-9308-1d974a7cb7e3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:41.139741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] The token cache is being invalidate because project c41aa54869d842f5b17a34e0d99364dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:41.140031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:41.140689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-803d109d-dcbf-4d88-9308-1d974a7cb7e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:41.142396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1166/2342] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:41 2022] DELETE /identity/v3/projects/c41aa54869d842f5b17a34e0d99364dc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:41.147785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e04622-9e03-48cf-9087-dfc73263ca65 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:41.148103 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e04622-9e03-48cf-9087-dfc73263ca65 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:41.148381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e04622-9e03-48cf-9087-dfc73263ca65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:41.167360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-82e04622-9e03-48cf-9087-dfc73263ca65 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:41.169369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-82e04622-9e03-48cf-9087-dfc73263ca65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:41.215254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:41.232860 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1177/2343] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:41.296337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd3776-e40d-4c5c-b04b-5b1e93b907d6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:41.296629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd3776-e40d-4c5c-b04b-5b1e93b907d6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:41.296907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd3776-e40d-4c5c-b04b-5b1e93b907d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:41.315547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ddfd3776-e40d-4c5c-b04b-5b1e93b907d6 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:41.316749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ddfd3776-e40d-4c5c-b04b-5b1e93b907d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:41.324638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1167/2344] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:41.329446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e04622-9e03-48cf-9087-dfc73263ca65 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:41.339502 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e04622-9e03-48cf-9087-dfc73263ca65 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:41.345505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8955fe-5d33-46e7-aa59-bbcfa656c248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:41.346236 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8955fe-5d33-46e7-aa59-bbcfa656c248 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:41.346526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8955fe-5d33-46e7-aa59-bbcfa656c248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:41.346814 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8955fe-5d33-46e7-aa59-bbcfa656c248 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:41.347557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8955fe-5d33-46e7-aa59-bbcfa656c248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:36:41.347899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8955fe-5d33-46e7-aa59-bbcfa656c248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:41.349537 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8955fe-5d33-46e7-aa59-bbcfa656c248 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:41.362940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1178/2345] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:36:41.367346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd3776-e40d-4c5c-b04b-5b1e93b907d6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:41.377337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd3776-e40d-4c5c-b04b-5b1e93b907d6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:41.382137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85408457-d276-48e1-81ed-da7b113b689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:41.382805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85408457-d276-48e1-81ed-da7b113b689f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:41.383090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85408457-d276-48e1-81ed-da7b113b689f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:41.383367 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85408457-d276-48e1-81ed-da7b113b689f admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:41.384078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85408457-d276-48e1-81ed-da7b113b689f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:41.386144 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85408457-d276-48e1-81ed-da7b113b689f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:41.414580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1168/2346] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:41 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:36:41.418966 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8955fe-5d33-46e7-aa59-bbcfa656c248 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:41.429263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8955fe-5d33-46e7-aa59-bbcfa656c248 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:41.488995 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbcc64-ee65-4b74-8675-31af8d4a12c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:41.489682 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbcc64-ee65-4b74-8675-31af8d4a12c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:41.489992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbcc64-ee65-4b74-8675-31af8d4a12c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:41.490284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbcc64-ee65-4b74-8675-31af8d4a12c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:41.490978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbcc64-ee65-4b74-8675-31af8d4a12c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:41.492923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbcc64-ee65-4b74-8675-31af8d4a12c3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:41.516032 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1179/2347] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:41 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:41.517188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85408457-d276-48e1-81ed-da7b113b689f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:41.527753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85408457-d276-48e1-81ed-da7b113b689f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:41.532378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2fdcd-a29a-4c7c-b92b-561325aad726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:41.533037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2fdcd-a29a-4c7c-b92b-561325aad726 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:41.533320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2fdcd-a29a-4c7c-b92b-561325aad726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:41.533595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2fdcd-a29a-4c7c-b92b-561325aad726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:41.534349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2fdcd-a29a-4c7c-b92b-561325aad726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:41.535971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2fdcd-a29a-4c7c-b92b-561325aad726 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:41.546817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1169/2348] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:36:41.550903 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbcc64-ee65-4b74-8675-31af8d4a12c3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:41.561693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbcc64-ee65-4b74-8675-31af8d4a12c3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:41.566453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba9d21-9ffb-41e4-8d1f-8f0b938ba8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:41.567112 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba9d21-9ffb-41e4-8d1f-8f0b938ba8de admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:41.567401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba9d21-9ffb-41e4-8d1f-8f0b938ba8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:41.567685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba9d21-9ffb-41e4-8d1f-8f0b938ba8de admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:41.568412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba9d21-9ffb-41e4-8d1f-8f0b938ba8de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:41.570037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba9d21-9ffb-41e4-8d1f-8f0b938ba8de admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:41.580308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1180/2349] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:41.584481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2fdcd-a29a-4c7c-b92b-561325aad726 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:41.594514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2fdcd-a29a-4c7c-b92b-561325aad726 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:41.599115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3959b-7ded-475c-af7d-db8e022a3be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:41.599987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3959b-7ded-475c-af7d-db8e022a3be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:41.600271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3959b-7ded-475c-af7d-db8e022a3be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:41.600543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3959b-7ded-475c-af7d-db8e022a3be0 admin admin] PATH_INFO: `/v3/projects/d55d9c640bee42f68d83fcfefbb07dc1/users/a1bdd163a16d455e964ff512fd1d4d96/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:41.615530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3959b-7ded-475c-af7d-db8e022a3be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d55d9c640bee42f68d83fcfefbb07dc1, user_id=a1bdd163a16d455e964ff512fd1d4d96, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:41.618142 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3959b-7ded-475c-af7d-db8e022a3be0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:41.635690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1170/2350] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:41 2022] PUT /identity/v3/projects/d55d9c640bee42f68d83fcfefbb07dc1/users/a1bdd163a16d455e964ff512fd1d4d96/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:41.646065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d29b62-940e-47db-a9c6-d99efa5b0c59 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:41.646367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d29b62-940e-47db-a9c6-d99efa5b0c59 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:41.646655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d29b62-940e-47db-a9c6-d99efa5b0c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:41.674412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e1d29b62-940e-47db-a9c6-d99efa5b0c59 None None] MFA Rules not processed for user `a1bdd163a16d455e964ff512fd1d4d96`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:41.675646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e1d29b62-940e-47db-a9c6-d99efa5b0c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:41.740684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1181/2351] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:41.752277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3959b-7ded-475c-af7d-db8e022a3be0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:41.754051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cdf3959b-7ded-475c-af7d-db8e022a3be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:41.774192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:41.816765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3959b-7ded-475c-af7d-db8e022a3be0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:41.821270 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ffaf5-a52a-41a5-a9f0-91f8be039c4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:41.822226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ffaf5-a52a-41a5-a9f0-91f8be039c4e service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:41.823475 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ffaf5-a52a-41a5-a9f0-91f8be039c4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:41.823475 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ffaf5-a52a-41a5-a9f0-91f8be039c4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:41.834032 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ffaf5-a52a-41a5-a9f0-91f8be039c4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:41.839121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ffaf5-a52a-41a5-a9f0-91f8be039c4e service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:41.851011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1171/2352] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:44.903130 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d29b62-940e-47db-a9c6-d99efa5b0c59 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:44.949685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:44.966220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d29b62-940e-47db-a9c6-d99efa5b0c59 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:44.970743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:44.971614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:44.971894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:44.972166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] PATH_INFO: `/v3/users/a1bdd163a16d455e964ff512fd1d4d96` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:44.973335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1bdd163a16d455e964ff512fd1d4d96)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:44.975328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:45.020325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1bdd163a16d455e964ff512fd1d4d96'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:45.032769 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1bdd163a16d455e964ff512fd1d4d96'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:45.039650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1bdd163a16d455e964ff512fd1d4d96'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:45.040319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1bdd163a16d455e964ff512fd1d4d96'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:45.040935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1bdd163a16d455e964ff512fd1d4d96'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:45.056032 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1182/2353] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:36:44 2022] DELETE /identity/v3/users/a1bdd163a16d455e964ff512fd1d4d96 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:45.060564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a1883-3d9b-49ed-bf1d-0528eef95a3f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:45.061009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a1883-3d9b-49ed-bf1d-0528eef95a3f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:45.061134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a1883-3d9b-49ed-bf1d-0528eef95a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:45.078070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-817a1883-3d9b-49ed-bf1d-0528eef95a3f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:45.079293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-817a1883-3d9b-49ed-bf1d-0528eef95a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:45.137988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1172/2354] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:45.147706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:45.149099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:45.209568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9de95-8a5d-4122-b551-5c47c28b3d29 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:45.213185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed7290-1969-487e-8a27-79cca5970360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:45.213828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed7290-1969-487e-8a27-79cca5970360 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:45.214141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed7290-1969-487e-8a27-79cca5970360 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:45.214423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed7290-1969-487e-8a27-79cca5970360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:45.224489 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed7290-1969-487e-8a27-79cca5970360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:45.225911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed7290-1969-487e-8a27-79cca5970360 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:45.235725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1183/2355] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:45.549914 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a1883-3d9b-49ed-bf1d-0528eef95a3f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:45.551374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-817a1883-3d9b-49ed-bf1d-0528eef95a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:45.563929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a1883-3d9b-49ed-bf1d-0528eef95a3f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:45.568624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:45.569500 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:45.569790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:45.570151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] PATH_INFO: `/v3/projects/d55d9c640bee42f68d83fcfefbb07dc1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:45.571346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d55d9c640bee42f68d83fcfefbb07dc1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:45.573267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:45.620492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd55d9c640bee42f68d83fcfefbb07dc1', 'request_id': 'req-5c5a4822-3b4c-4574-ac3f-13d21edeca45'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:45.625932 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd55d9c640bee42f68d83fcfefbb07dc1', 'request_id': 'req-5c5a4822-3b4c-4574-ac3f-13d21edeca45'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:45.632021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd55d9c640bee42f68d83fcfefbb07dc1', 'request_id': 'req-5c5a4822-3b4c-4574-ac3f-13d21edeca45'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:45.643958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] The token cache is being invalidate because project d55d9c640bee42f68d83fcfefbb07dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:36:45.644241 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:45.644889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c5a4822-3b4c-4574-ac3f-13d21edeca45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:36:45.646471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1173/2356] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:36:45 2022] DELETE /identity/v3/projects/d55d9c640bee42f68d83fcfefbb07dc1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:45.651865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab3b0c-d38a-4d5c-9580-6fa01862fe8f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:45.652161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab3b0c-d38a-4d5c-9580-6fa01862fe8f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:45.652446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab3b0c-d38a-4d5c-9580-6fa01862fe8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:45.671529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ebab3b0c-d38a-4d5c-9580-6fa01862fe8f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:45.672655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ebab3b0c-d38a-4d5c-9580-6fa01862fe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:45.732532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1184/2357] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:36:45.741133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff8450-2f7f-4a82-a60a-20ad8d36c6b1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:45.741502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff8450-2f7f-4a82-a60a-20ad8d36c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:45.741786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff8450-2f7f-4a82-a60a-20ad8d36c6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:45.760671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0dff8450-2f7f-4a82-a60a-20ad8d36c6b1 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:45.761889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0dff8450-2f7f-4a82-a60a-20ad8d36c6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:45.770981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1174/2358] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:45.775452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab3b0c-d38a-4d5c-9580-6fa01862fe8f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:45.785827 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab3b0c-d38a-4d5c-9580-6fa01862fe8f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:45.790588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce920f1-673c-4786-aa70-cbb627b5d530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:45.791255 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce920f1-673c-4786-aa70-cbb627b5d530 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:45.791652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce920f1-673c-4786-aa70-cbb627b5d530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:45.791935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce920f1-673c-4786-aa70-cbb627b5d530 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:45.792666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce920f1-673c-4786-aa70-cbb627b5d530 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:36:45.792974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce920f1-673c-4786-aa70-cbb627b5d530 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:45.795810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce920f1-673c-4786-aa70-cbb627b5d530 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:45.798746 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:45.814275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1185/2359] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:36:45.814966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff8450-2f7f-4a82-a60a-20ad8d36c6b1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:45.824292 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff8450-2f7f-4a82-a60a-20ad8d36c6b1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:45.829536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff41053-001a-40df-b28c-c0c73ab0611d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:45.830228 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41053-001a-40df-b28c-c0c73ab0611d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:45.830512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41053-001a-40df-b28c-c0c73ab0611d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:45.830790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41053-001a-40df-b28c-c0c73ab0611d admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:45.831479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff41053-001a-40df-b28c-c0c73ab0611d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:45.833410 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff41053-001a-40df-b28c-c0c73ab0611d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:45.861291 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1175/2360] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:45 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:36:45.865469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce920f1-673c-4786-aa70-cbb627b5d530 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:45.876550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce920f1-673c-4786-aa70-cbb627b5d530 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:45.937374 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14b11e-b5cc-4e03-81d4-44ad1b890bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:45.938061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14b11e-b5cc-4e03-81d4-44ad1b890bee admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:45.938269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14b11e-b5cc-4e03-81d4-44ad1b890bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:45.938550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14b11e-b5cc-4e03-81d4-44ad1b890bee admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:45.939251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14b11e-b5cc-4e03-81d4-44ad1b890bee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:45.941210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14b11e-b5cc-4e03-81d4-44ad1b890bee admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:45.966310 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1186/2361] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:45 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:45.968395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff41053-001a-40df-b28c-c0c73ab0611d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:45.978952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff41053-001a-40df-b28c-c0c73ab0611d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:45.987883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd79ec8-b9cd-46f4-9709-56003595644f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:45.988563 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd79ec8-b9cd-46f4-9709-56003595644f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:45.988846 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd79ec8-b9cd-46f4-9709-56003595644f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:45.989149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd79ec8-b9cd-46f4-9709-56003595644f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:45.989890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd79ec8-b9cd-46f4-9709-56003595644f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:45.991518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd79ec8-b9cd-46f4-9709-56003595644f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:46.004393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1176/2362] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:36:46.008201 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14b11e-b5cc-4e03-81d4-44ad1b890bee admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:46.018393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14b11e-b5cc-4e03-81d4-44ad1b890bee admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:46.024142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efe9c7-a8f7-4b83-b666-605c6fdb8c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:46.025069 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efe9c7-a8f7-4b83-b666-605c6fdb8c1a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:46.025355 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efe9c7-a8f7-4b83-b666-605c6fdb8c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:46.025633 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efe9c7-a8f7-4b83-b666-605c6fdb8c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:46.026399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efe9c7-a8f7-4b83-b666-605c6fdb8c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:46.028206 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efe9c7-a8f7-4b83-b666-605c6fdb8c1a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:46.039290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1187/2363] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:46.044475 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd79ec8-b9cd-46f4-9709-56003595644f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:46.053155 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:46.057680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd79ec8-b9cd-46f4-9709-56003595644f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:46.065519 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3149d41-5899-44e5-8589-5d36810c9193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:46.067131 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3149d41-5899-44e5-8589-5d36810c9193 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:46.067482 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3149d41-5899-44e5-8589-5d36810c9193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:46.067837 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3149d41-5899-44e5-8589-5d36810c9193 admin admin] PATH_INFO: `/v3/projects/fd3a6e00103d49038260d28a680a9c56/users/0f951e37e60145aba97bcd1ff8aad260/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:46.083190 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3149d41-5899-44e5-8589-5d36810c9193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3a6e00103d49038260d28a680a9c56, user_id=0f951e37e60145aba97bcd1ff8aad260, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:46.086044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3149d41-5899-44e5-8589-5d36810c9193 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:46.103281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1177/2364] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:46 2022] PUT /identity/v3/projects/fd3a6e00103d49038260d28a680a9c56/users/0f951e37e60145aba97bcd1ff8aad260/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:46.109673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18122e-fc41-47ba-82ff-2a24bb0173e6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:46.110023 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18122e-fc41-47ba-82ff-2a24bb0173e6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:46.110323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18122e-fc41-47ba-82ff-2a24bb0173e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:46.131573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0f18122e-fc41-47ba-82ff-2a24bb0173e6 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:46.132783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0f18122e-fc41-47ba-82ff-2a24bb0173e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:46.194751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1188/2365] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:36:46.204966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3149d41-5899-44e5-8589-5d36810c9193 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:46.206789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d3149d41-5899-44e5-8589-5d36810c9193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:46.265199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3149d41-5899-44e5-8589-5d36810c9193 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:46.269168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034cfb9-d4c2-437d-9fb6-425fa6d77f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:46.269832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034cfb9-d4c2-437d-9fb6-425fa6d77f0b service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:46.270160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034cfb9-d4c2-437d-9fb6-425fa6d77f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:46.270453 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034cfb9-d4c2-437d-9fb6-425fa6d77f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:46.280731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034cfb9-d4c2-437d-9fb6-425fa6d77f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:46.282227 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034cfb9-d4c2-437d-9fb6-425fa6d77f0b service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:46.291887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1178/2366] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:50.756505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a698b-3242-4633-9747-a437cdd8f1d9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:50.757010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a698b-3242-4633-9747-a437cdd8f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:50.757109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a698b-3242-4633-9747-a437cdd8f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:50.788171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-db3a698b-3242-4633-9747-a437cdd8f1d9 None None] MFA Rules not processed for user `0f951e37e60145aba97bcd1ff8aad260`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:50.789358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-db3a698b-3242-4633-9747-a437cdd8f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:50.889013 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1189/2367] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:50.893500 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034cfb9-d4c2-437d-9fb6-425fa6d77f0b service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:50.909667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034cfb9-d4c2-437d-9fb6-425fa6d77f0b service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:50.915025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d597cd8-3434-4b58-b36b-cb744434f89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:50.915707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d597cd8-3434-4b58-b36b-cb744434f89b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:50.915995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d597cd8-3434-4b58-b36b-cb744434f89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:50.916281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d597cd8-3434-4b58-b36b-cb744434f89b admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:50.917001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d597cd8-3434-4b58-b36b-cb744434f89b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:50.919058 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d597cd8-3434-4b58-b36b-cb744434f89b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:50.957871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1179/2368] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:36:50 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:50.959151 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a698b-3242-4633-9747-a437cdd8f1d9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:50.967729 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:50.975978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a698b-3242-4633-9747-a437cdd8f1d9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:51.049181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eec10b-7194-458c-bb8b-1f7888929bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:51.049880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec10b-7194-458c-bb8b-1f7888929bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:51.050306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec10b-7194-458c-bb8b-1f7888929bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:51.050574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec10b-7194-458c-bb8b-1f7888929bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:51.051296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eec10b-7194-458c-bb8b-1f7888929bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:51.053437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eec10b-7194-458c-bb8b-1f7888929bc5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:51.078070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1190/2369] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:36:50 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:36:51.081783 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d597cd8-3434-4b58-b36b-cb744434f89b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:51.096378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d597cd8-3434-4b58-b36b-cb744434f89b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:51.101693 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918feb8-3b50-47fc-8f60-af5cf9091d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:51.102681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918feb8-3b50-47fc-8f60-af5cf9091d9e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:51.103189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918feb8-3b50-47fc-8f60-af5cf9091d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:51.103687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918feb8-3b50-47fc-8f60-af5cf9091d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:51.104667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918feb8-3b50-47fc-8f60-af5cf9091d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:51.106607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918feb8-3b50-47fc-8f60-af5cf9091d9e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:51.121108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1180/2370] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:51.126189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eec10b-7194-458c-bb8b-1f7888929bc5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:51.137027 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eec10b-7194-458c-bb8b-1f7888929bc5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:51.142136 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebab29-5e29-4805-a39f-829af6f6be86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:51.143149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebab29-5e29-4805-a39f-829af6f6be86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:51.143525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebab29-5e29-4805-a39f-829af6f6be86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:51.143875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebab29-5e29-4805-a39f-829af6f6be86 admin admin] PATH_INFO: `/v3/projects/f188a0b484cc4781953e86161e579322/users/99c87dbc5a9346ff80537975a92fa4f4/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:51.160592 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebab29-5e29-4805-a39f-829af6f6be86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f188a0b484cc4781953e86161e579322, user_id=99c87dbc5a9346ff80537975a92fa4f4, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:51.163317 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebab29-5e29-4805-a39f-829af6f6be86 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:51.177580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1191/2371] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:51 2022] PUT /identity/v3/projects/f188a0b484cc4781953e86161e579322/users/99c87dbc5a9346ff80537975a92fa4f4/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:36:51.181644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918feb8-3b50-47fc-8f60-af5cf9091d9e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:51.193397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918feb8-3b50-47fc-8f60-af5cf9091d9e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:51.259898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13717a99-d368-4c3c-8b6e-10106a633bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:51.260591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13717a99-d368-4c3c-8b6e-10106a633bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:51.260876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13717a99-d368-4c3c-8b6e-10106a633bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:51.261155 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13717a99-d368-4c3c-8b6e-10106a633bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:51.262038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13717a99-d368-4c3c-8b6e-10106a633bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:51.264784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13717a99-d368-4c3c-8b6e-10106a633bc7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:51.276562 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1181/2372] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:36:51.280747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebab29-5e29-4805-a39f-829af6f6be86 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:51.291665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebab29-5e29-4805-a39f-829af6f6be86 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:51.299305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b7e73-4af2-49f3-9349-606d432b28fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:51.300215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b7e73-4af2-49f3-9349-606d432b28fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:51.300503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b7e73-4af2-49f3-9349-606d432b28fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:51.300784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b7e73-4af2-49f3-9349-606d432b28fe admin admin] PATH_INFO: `/v3/projects/f188a0b484cc4781953e86161e579322/users/99c87dbc5a9346ff80537975a92fa4f4/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:51.302831 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b7e73-4af2-49f3-9349-606d432b28fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f188a0b484cc4781953e86161e579322, user_id=99c87dbc5a9346ff80537975a92fa4f4, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:51.307751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b7e73-4af2-49f3-9349-606d432b28fe admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:51.327262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1192/2373] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:36:51 2022] PUT /identity/v3/projects/f188a0b484cc4781953e86161e579322/users/99c87dbc5a9346ff80537975a92fa4f4/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:36:52.245036 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b558aad-4497-4220-b6f5-56a8ee22b994 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:52.245530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b558aad-4497-4220-b6f5-56a8ee22b994 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:52.245637 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b558aad-4497-4220-b6f5-56a8ee22b994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:52.279787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3b558aad-4497-4220-b6f5-56a8ee22b994 None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:52.281625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3b558aad-4497-4220-b6f5-56a8ee22b994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:52.330371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:52.347249 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1182/2374] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:36:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:52.404652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b7e73-4af2-49f3-9349-606d432b28fe admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:52.406532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e03b7e73-4af2-49f3-9349-606d432b28fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:52.505402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b7e73-4af2-49f3-9349-606d432b28fe admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:52.513176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8ac05-457b-4d9f-b96e-44c8e1a84cb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:52.514271 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8ac05-457b-4d9f-b96e-44c8e1a84cb0 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:52.514741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8ac05-457b-4d9f-b96e-44c8e1a84cb0 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:52.515239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8ac05-457b-4d9f-b96e-44c8e1a84cb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:52.526726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8ac05-457b-4d9f-b96e-44c8e1a84cb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:52.528602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8ac05-457b-4d9f-b96e-44c8e1a84cb0 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:52.538805 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1193/2375] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:36:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:36:57.446710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635b478-ba29-4e8e-a033-9c32eb376d9f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:57.447184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635b478-ba29-4e8e-a033-9c32eb376d9f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:57.447319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635b478-ba29-4e8e-a033-9c32eb376d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:57.480497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0635b478-ba29-4e8e-a033-9c32eb376d9f None None] MFA Rules not processed for user `99c87dbc5a9346ff80537975a92fa4f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:57.483627 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0635b478-ba29-4e8e-a033-9c32eb376d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:57.595034 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1183/2376] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:36:57.601683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8ac05-457b-4d9f-b96e-44c8e1a84cb0 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:36:57.603891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-eed8ac05-457b-4d9f-b96e-44c8e1a84cb0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:57.725260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:36:57.791505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8ac05-457b-4d9f-b96e-44c8e1a84cb0 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:36:57.799969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac0737-1fad-4b3a-8fe9-48d972d7f19e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:36:57.800830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac0737-1fad-4b3a-8fe9-48d972d7f19e service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:57.801557 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac0737-1fad-4b3a-8fe9-48d972d7f19e service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:57.803535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac0737-1fad-4b3a-8fe9-48d972d7f19e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:57.920386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac0737-1fad-4b3a-8fe9-48d972d7f19e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:36:57.922388 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac0737-1fad-4b3a-8fe9-48d972d7f19e service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:36:57.936149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1194/2377] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:36:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:36:59.562043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c537ba4-2b51-4798-94a8-e176989e36b3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:36:59.562591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c537ba4-2b51-4798-94a8-e176989e36b3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:36:59.562810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c537ba4-2b51-4798-94a8-e176989e36b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:36:59.587167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8c537ba4-2b51-4798-94a8-e176989e36b3 None None] MFA Rules not processed for user `0f951e37e60145aba97bcd1ff8aad260`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:36:59.588426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8c537ba4-2b51-4798-94a8-e176989e36b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:36:59.596596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1184/2378] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:36:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:07.930717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=35 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:09.580313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=41 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:12.851231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26447bf6-eef0-43e5-ab43-c7825afa307f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:12.851774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26447bf6-eef0-43e5-ab43-c7825afa307f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:12.851774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26447bf6-eef0-43e5-ab43-c7825afa307f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:12.869653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-26447bf6-eef0-43e5-ab43-c7825afa307f None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:12.870882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-26447bf6-eef0-43e5-ab43-c7825afa307f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:12.878804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1195/2379] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:37:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:12.890766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c537ba4-2b51-4798-94a8-e176989e36b3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:12.902732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c537ba4-2b51-4798-94a8-e176989e36b3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:12.908179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd478aa-154f-4cf0-80f7-51b72454246c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:12.908848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd478aa-154f-4cf0-80f7-51b72454246c service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:12.909134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd478aa-154f-4cf0-80f7-51b72454246c service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:12.909414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd478aa-154f-4cf0-80f7-51b72454246c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:12.917547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd478aa-154f-4cf0-80f7-51b72454246c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:12.919172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd478aa-154f-4cf0-80f7-51b72454246c service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:12.927964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1185/2380] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:37:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 06:37:22.864621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:22.924667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:29.380635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf5fc0-e029-4ef3-a648-9ba1ca814f72 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:29.381160 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf5fc0-e029-4ef3-a648-9ba1ca814f72 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:29.381353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf5fc0-e029-4ef3-a648-9ba1ca814f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:29.404829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2edf5fc0-e029-4ef3-a648-9ba1ca814f72 None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:29.406264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2edf5fc0-e029-4ef3-a648-9ba1ca814f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:29.413663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1196/2381] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jul 21 06:37:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:29.426496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd478aa-154f-4cf0-80f7-51b72454246c service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:29.437025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd478aa-154f-4cf0-80f7-51b72454246c service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:29.443340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdde969-194e-46c4-a915-5a787d8fe970 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:29.443982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdde969-194e-46c4-a915-5a787d8fe970 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:29.444268 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdde969-194e-46c4-a915-5a787d8fe970 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:29.444547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdde969-194e-46c4-a915-5a787d8fe970 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:29.454066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdde969-194e-46c4-a915-5a787d8fe970 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:29.455644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdde969-194e-46c4-a915-5a787d8fe970 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:29.463788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1186/2382] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:37:38.100942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf5fc0-e029-4ef3-a648-9ba1ca814f72 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:38.102419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2edf5fc0-e029-4ef3-a648-9ba1ca814f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:38.160259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf5fc0-e029-4ef3-a648-9ba1ca814f72 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:38.163951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95f016-7d1c-4486-bd47-81d257af37df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:38.164601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95f016-7d1c-4486-bd47-81d257af37df service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:38.164886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95f016-7d1c-4486-bd47-81d257af37df service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:38.165167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95f016-7d1c-4486-bd47-81d257af37df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:38.166940 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1a95f016-7d1c-4486-bd47-81d257af37df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:38.229415 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95f016-7d1c-4486-bd47-81d257af37df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:38.230878 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95f016-7d1c-4486-bd47-81d257af37df service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:38.240558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1197/2383] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:37:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:37:39.460627 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:40.716345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdde969-194e-46c4-a915-5a787d8fe970 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:40.726258 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdde969-194e-46c4-a915-5a787d8fe970 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:40.729800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d399770-2b1c-41b9-9fb9-efaf67bd3e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:40.730471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d399770-2b1c-41b9-9fb9-efaf67bd3e7e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:40.730754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d399770-2b1c-41b9-9fb9-efaf67bd3e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:40.731030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d399770-2b1c-41b9-9fb9-efaf67bd3e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:40.741109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d399770-2b1c-41b9-9fb9-efaf67bd3e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:40.742440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d399770-2b1c-41b9-9fb9-efaf67bd3e7e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:40.752951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1187/2384] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:37:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:37:41.876606 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95f016-7d1c-4486-bd47-81d257af37df service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:41.878174 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1a95f016-7d1c-4486-bd47-81d257af37df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:41.899224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95f016-7d1c-4486-bd47-81d257af37df service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:41.910564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a79ef0-b63d-4543-a36d-0c19c10051f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:41.911519 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a79ef0-b63d-4543-a36d-0c19c10051f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:41.911806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a79ef0-b63d-4543-a36d-0c19c10051f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:41.912324 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a79ef0-b63d-4543-a36d-0c19c10051f6 admin admin] PATH_INFO: `/v3/users/8561734cf36e47b5ab2d7242076437ff` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:41.913699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a79ef0-b63d-4543-a36d-0c19c10051f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8561734cf36e47b5ab2d7242076437ff)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:41.916593 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a79ef0-b63d-4543-a36d-0c19c10051f6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:41.990366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c9a79ef0-b63d-4543-a36d-0c19c10051f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8561734cf36e47b5ab2d7242076437ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:42.004706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c9a79ef0-b63d-4543-a36d-0c19c10051f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8561734cf36e47b5ab2d7242076437ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:42.014412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c9a79ef0-b63d-4543-a36d-0c19c10051f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8561734cf36e47b5ab2d7242076437ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:42.015803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c9a79ef0-b63d-4543-a36d-0c19c10051f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8561734cf36e47b5ab2d7242076437ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:42.016421 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c9a79ef0-b63d-4543-a36d-0c19c10051f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8561734cf36e47b5ab2d7242076437ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:42.032241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1198/2385] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:37:41 2022] DELETE /identity/v3/users/8561734cf36e47b5ab2d7242076437ff => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:42.231622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d399770-2b1c-41b9-9fb9-efaf67bd3e7e service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:42.244776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d399770-2b1c-41b9-9fb9-efaf67bd3e7e service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:42.329960 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:42.330871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:42.331223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:42.331574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] PATH_INFO: `/v3/projects/b2843a54fec340e19485e7b821fc071a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:42.332877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2843a54fec340e19485e7b821fc071a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:42.334912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:42.391633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2843a54fec340e19485e7b821fc071a', 'request_id': 'req-68bae26d-4945-4291-8386-dba77e649d67'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:42.398076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2843a54fec340e19485e7b821fc071a', 'request_id': 'req-68bae26d-4945-4291-8386-dba77e649d67'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:42.409204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2843a54fec340e19485e7b821fc071a', 'request_id': 'req-68bae26d-4945-4291-8386-dba77e649d67'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:42.422416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] The token cache is being invalidate because project b2843a54fec340e19485e7b821fc071a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:37:42.422811 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:42.423507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-68bae26d-4945-4291-8386-dba77e649d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:42.428650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1188/2386] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:37:42 2022] DELETE /identity/v3/projects/b2843a54fec340e19485e7b821fc071a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:42.430448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18399211-806c-4cce-8e9b-d3f45d7c223f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:42.430744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18399211-806c-4cce-8e9b-d3f45d7c223f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:42.431061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18399211-806c-4cce-8e9b-d3f45d7c223f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:42.450564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-18399211-806c-4cce-8e9b-d3f45d7c223f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:42.451771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-18399211-806c-4cce-8e9b-d3f45d7c223f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:42.463893 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:42.529977 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1199/2387] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:42.537569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bea6a9-b8e3-4496-b4f7-a050f78c657c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:42.537853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bea6a9-b8e3-4496-b4f7-a050f78c657c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:42.538185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bea6a9-b8e3-4496-b4f7-a050f78c657c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:42.558111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f6bea6a9-b8e3-4496-b4f7-a050f78c657c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:42.559599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f6bea6a9-b8e3-4496-b4f7-a050f78c657c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:42.568947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1189/2388] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:42.573767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18399211-806c-4cce-8e9b-d3f45d7c223f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:42.584799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18399211-806c-4cce-8e9b-d3f45d7c223f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:42.592372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c848f-9a6b-44c3-9f47-646c2f5e7a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:42.594373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c848f-9a6b-44c3-9f47-646c2f5e7a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:42.594666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c848f-9a6b-44c3-9f47-646c2f5e7a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:42.594950 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c848f-9a6b-44c3-9f47-646c2f5e7a30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:42.595710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c848f-9a6b-44c3-9f47-646c2f5e7a30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:37:42.596025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c848f-9a6b-44c3-9f47-646c2f5e7a30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:42.597670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c848f-9a6b-44c3-9f47-646c2f5e7a30 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:42.611108 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1200/2389] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:37:42.615633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bea6a9-b8e3-4496-b4f7-a050f78c657c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:42.628122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bea6a9-b8e3-4496-b4f7-a050f78c657c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:42.635156 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa194f4-92cf-432b-b0d6-d08fe4f92619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:42.636048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa194f4-92cf-432b-b0d6-d08fe4f92619 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:42.637114 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa194f4-92cf-432b-b0d6-d08fe4f92619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:42.637714 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa194f4-92cf-432b-b0d6-d08fe4f92619 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:42.638882 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa194f4-92cf-432b-b0d6-d08fe4f92619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:42.642814 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa194f4-92cf-432b-b0d6-d08fe4f92619 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:42.684180 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1190/2390] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:37:42 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:37:42.685425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c848f-9a6b-44c3-9f47-646c2f5e7a30 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:42.701218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c848f-9a6b-44c3-9f47-646c2f5e7a30 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:42.787895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe02fdd-b99c-4d53-980b-e0bf6e4956ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:42.787895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe02fdd-b99c-4d53-980b-e0bf6e4956ef admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:42.787895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe02fdd-b99c-4d53-980b-e0bf6e4956ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:42.787895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe02fdd-b99c-4d53-980b-e0bf6e4956ef admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:42.788259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe02fdd-b99c-4d53-980b-e0bf6e4956ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:42.788259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe02fdd-b99c-4d53-980b-e0bf6e4956ef admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:42.808466 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1201/2391] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:37:42 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:37:42.809709 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa194f4-92cf-432b-b0d6-d08fe4f92619 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:42.820639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa194f4-92cf-432b-b0d6-d08fe4f92619 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:42.825288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82df1ee-8044-4300-b084-198b140d2204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:42.825981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82df1ee-8044-4300-b084-198b140d2204 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:42.826272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82df1ee-8044-4300-b084-198b140d2204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:42.826555 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82df1ee-8044-4300-b084-198b140d2204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:42.828108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82df1ee-8044-4300-b084-198b140d2204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:42.829864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82df1ee-8044-4300-b084-198b140d2204 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:42.840949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1191/2392] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:37:42.844873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe02fdd-b99c-4d53-980b-e0bf6e4956ef admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:42.855232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe02fdd-b99c-4d53-980b-e0bf6e4956ef admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:42.860577 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca341e-fa8b-4440-a5df-25e5d32ca67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:42.861245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca341e-fa8b-4440-a5df-25e5d32ca67b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:42.861531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca341e-fa8b-4440-a5df-25e5d32ca67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:42.861812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca341e-fa8b-4440-a5df-25e5d32ca67b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:42.862573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca341e-fa8b-4440-a5df-25e5d32ca67b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:42.864168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca341e-fa8b-4440-a5df-25e5d32ca67b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:42.874696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1202/2393] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:37:42.879122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82df1ee-8044-4300-b084-198b140d2204 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:42.889570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82df1ee-8044-4300-b084-198b140d2204 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:42.895138 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cb97f-34da-49bb-aef3-47ebed148919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:42.896038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cb97f-34da-49bb-aef3-47ebed148919 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:42.896322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cb97f-34da-49bb-aef3-47ebed148919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:42.896601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cb97f-34da-49bb-aef3-47ebed148919 admin admin] PATH_INFO: `/v3/projects/e927311b947c48629d1ebd6c29ba7d50/users/d4e425d00b4a4f1b9043c91d23ed9331/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:42.912995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cb97f-34da-49bb-aef3-47ebed148919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e927311b947c48629d1ebd6c29ba7d50, user_id=d4e425d00b4a4f1b9043c91d23ed9331, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:42.915746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cb97f-34da-49bb-aef3-47ebed148919 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:42.930229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1192/2394] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:37:42 2022] PUT /identity/v3/projects/e927311b947c48629d1ebd6c29ba7d50/users/d4e425d00b4a4f1b9043c91d23ed9331/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:37:42.940752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155def6-a1de-45ac-b650-46b79f4d8732 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:42.941593 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155def6-a1de-45ac-b650-46b79f4d8732 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:42.941884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155def6-a1de-45ac-b650-46b79f4d8732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:42.968416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7155def6-a1de-45ac-b650-46b79f4d8732 None None] MFA Rules not processed for user `d4e425d00b4a4f1b9043c91d23ed9331`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:42.969644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7155def6-a1de-45ac-b650-46b79f4d8732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:43.037351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1203/2395] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:43.050667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cb97f-34da-49bb-aef3-47ebed148919 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:43.052423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-884cb97f-34da-49bb-aef3-47ebed148919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:43.067710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:43.132630 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cb97f-34da-49bb-aef3-47ebed148919 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:43.138211 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdce9a9-254d-400f-b7eb-12d07147cf8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:43.138873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdce9a9-254d-400f-b7eb-12d07147cf8f service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:43.139160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdce9a9-254d-400f-b7eb-12d07147cf8f service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:43.139461 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdce9a9-254d-400f-b7eb-12d07147cf8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:43.147683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdce9a9-254d-400f-b7eb-12d07147cf8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:43.149299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdce9a9-254d-400f-b7eb-12d07147cf8f service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:43.159348 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1193/2396] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:37:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:37:45.899437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155def6-a1de-45ac-b650-46b79f4d8732 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:45.909181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:45.971210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155def6-a1de-45ac-b650-46b79f4d8732 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:45.976134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:45.977291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:45.977291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:45.977578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] PATH_INFO: `/v3/users/d4e425d00b4a4f1b9043c91d23ed9331` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:45.978841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4e425d00b4a4f1b9043c91d23ed9331)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:45.980859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:46.032121 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4e425d00b4a4f1b9043c91d23ed9331'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:46.046759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4e425d00b4a4f1b9043c91d23ed9331'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:46.055360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4e425d00b4a4f1b9043c91d23ed9331'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:46.056318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4e425d00b4a4f1b9043c91d23ed9331'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:46.057077 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4e425d00b4a4f1b9043c91d23ed9331'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:46.071915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1204/2397] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:37:45 2022] DELETE /identity/v3/users/d4e425d00b4a4f1b9043c91d23ed9331 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:46.076484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeabd1a-cf82-4dd2-998c-04c80a96ee43 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:46.076968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeabd1a-cf82-4dd2-998c-04c80a96ee43 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:46.077085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeabd1a-cf82-4dd2-998c-04c80a96ee43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:46.096510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-faeabd1a-cf82-4dd2-998c-04c80a96ee43 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:46.097720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-faeabd1a-cf82-4dd2-998c-04c80a96ee43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:46.186518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1194/2398] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:46.197090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:46.198680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:46.273644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e018a-bc55-4250-9ae7-303d66b2b65f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:46.277772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5025e-1dde-4408-8ed9-359dcc4c3640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:46.278942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5025e-1dde-4408-8ed9-359dcc4c3640 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:46.278942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5025e-1dde-4408-8ed9-359dcc4c3640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:46.279311 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5025e-1dde-4408-8ed9-359dcc4c3640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:46.289395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5025e-1dde-4408-8ed9-359dcc4c3640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:46.291032 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5025e-1dde-4408-8ed9-359dcc4c3640 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:46.304093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1205/2399] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:37:46.765259 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeabd1a-cf82-4dd2-998c-04c80a96ee43 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:46.766720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-faeabd1a-cf82-4dd2-998c-04c80a96ee43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:46.792049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeabd1a-cf82-4dd2-998c-04c80a96ee43 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:46.799180 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:46.800505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:46.800797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:46.801083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] PATH_INFO: `/v3/projects/e927311b947c48629d1ebd6c29ba7d50` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:46.802361 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e927311b947c48629d1ebd6c29ba7d50)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:46.804294 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:46.857449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e927311b947c48629d1ebd6c29ba7d50', 'request_id': 'req-94616430-1d9e-41dc-85a9-bc2131409793'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:46.863917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e927311b947c48629d1ebd6c29ba7d50', 'request_id': 'req-94616430-1d9e-41dc-85a9-bc2131409793'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:46.870317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e927311b947c48629d1ebd6c29ba7d50', 'request_id': 'req-94616430-1d9e-41dc-85a9-bc2131409793'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:46.883845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] The token cache is being invalidate because project e927311b947c48629d1ebd6c29ba7d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:37:46.884140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:46.884775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94616430-1d9e-41dc-85a9-bc2131409793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:46.886527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1195/2400] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:37:46 2022] DELETE /identity/v3/projects/e927311b947c48629d1ebd6c29ba7d50 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:46.891581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d105a-e436-41a4-86fe-df8ad30bf59b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:46.891883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d105a-e436-41a4-86fe-df8ad30bf59b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:46.892170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d105a-e436-41a4-86fe-df8ad30bf59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:46.919706 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-da1d105a-e436-41a4-86fe-df8ad30bf59b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:46.922506 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-da1d105a-e436-41a4-86fe-df8ad30bf59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:46.957687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:46.998139 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1206/2401] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:47.015572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e38533-61bd-4644-b895-2109054367b0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.015857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e38533-61bd-4644-b895-2109054367b0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.016153 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e38533-61bd-4644-b895-2109054367b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.034608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-66e38533-61bd-4644-b895-2109054367b0 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:47.035813 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-66e38533-61bd-4644-b895-2109054367b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:47.051657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1196/2402] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:47.057411 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d105a-e436-41a4-86fe-df8ad30bf59b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.067803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d105a-e436-41a4-86fe-df8ad30bf59b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.072419 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63e65e-d8a7-448c-96cc-16468e309b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.073081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63e65e-d8a7-448c-96cc-16468e309b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.073370 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63e65e-d8a7-448c-96cc-16468e309b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.073654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63e65e-d8a7-448c-96cc-16468e309b90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.074426 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63e65e-d8a7-448c-96cc-16468e309b90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:37:47.074741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63e65e-d8a7-448c-96cc-16468e309b90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.076330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63e65e-d8a7-448c-96cc-16468e309b90 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.088798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1207/2403] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:37:47.093309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e38533-61bd-4644-b895-2109054367b0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.102913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e38533-61bd-4644-b895-2109054367b0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.107690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019daf1-3b50-4757-9e4e-0ef44302d88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.108342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019daf1-3b50-4757-9e4e-0ef44302d88b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.108633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019daf1-3b50-4757-9e4e-0ef44302d88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.108911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019daf1-3b50-4757-9e4e-0ef44302d88b admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.109598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019daf1-3b50-4757-9e4e-0ef44302d88b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.111546 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019daf1-3b50-4757-9e4e-0ef44302d88b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.125262 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:47.138433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1197/2404] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:37:47 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:37:47.143154 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63e65e-d8a7-448c-96cc-16468e309b90 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.153379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63e65e-d8a7-448c-96cc-16468e309b90 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.211444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b80ac-d63b-4008-82fc-31e09865c033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.212224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b80ac-d63b-4008-82fc-31e09865c033 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.212546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b80ac-d63b-4008-82fc-31e09865c033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.212885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b80ac-d63b-4008-82fc-31e09865c033 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.213684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b80ac-d63b-4008-82fc-31e09865c033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.215961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b80ac-d63b-4008-82fc-31e09865c033 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.236656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1208/2405] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:37:47 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:37:47.240891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019daf1-3b50-4757-9e4e-0ef44302d88b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.253377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019daf1-3b50-4757-9e4e-0ef44302d88b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.258693 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00361b14-bcf7-47c1-8358-ccdf5bfb7cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.259378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00361b14-bcf7-47c1-8358-ccdf5bfb7cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.259641 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00361b14-bcf7-47c1-8358-ccdf5bfb7cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.259917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00361b14-bcf7-47c1-8358-ccdf5bfb7cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.260706 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00361b14-bcf7-47c1-8358-ccdf5bfb7cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.262389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00361b14-bcf7-47c1-8358-ccdf5bfb7cb1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.274639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1198/2406] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:37:47.280418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b80ac-d63b-4008-82fc-31e09865c033 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.290455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b80ac-d63b-4008-82fc-31e09865c033 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.294979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ee200-2f7f-45cb-8e8a-261ee108e082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.295630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ee200-2f7f-45cb-8e8a-261ee108e082 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.295911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ee200-2f7f-45cb-8e8a-261ee108e082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.296213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ee200-2f7f-45cb-8e8a-261ee108e082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.297092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ee200-2f7f-45cb-8e8a-261ee108e082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.298730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ee200-2f7f-45cb-8e8a-261ee108e082 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.308999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1209/2407] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:37:47.313917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00361b14-bcf7-47c1-8358-ccdf5bfb7cb1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.324496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00361b14-bcf7-47c1-8358-ccdf5bfb7cb1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.330093 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a488257-a79d-45b1-b393-43768256bb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.330985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a488257-a79d-45b1-b393-43768256bb39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.331267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a488257-a79d-45b1-b393-43768256bb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.331543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a488257-a79d-45b1-b393-43768256bb39 admin admin] PATH_INFO: `/v3/projects/0eaba88de9e349a7852a6c6a92650fc7/users/ec7b14cd0ec84062bf7c235a54838e95/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.346719 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a488257-a79d-45b1-b393-43768256bb39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eaba88de9e349a7852a6c6a92650fc7, user_id=ec7b14cd0ec84062bf7c235a54838e95, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.349340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a488257-a79d-45b1-b393-43768256bb39 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.362474 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1199/2408] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:37:47 2022] PUT /identity/v3/projects/0eaba88de9e349a7852a6c6a92650fc7/users/ec7b14cd0ec84062bf7c235a54838e95/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:47.372787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e108a-5368-4c32-8619-812481ed6820 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.373153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e108a-5368-4c32-8619-812481ed6820 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.373454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e108a-5368-4c32-8619-812481ed6820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.400341 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e23e108a-5368-4c32-8619-812481ed6820 None None] MFA Rules not processed for user `ec7b14cd0ec84062bf7c235a54838e95`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:47.401789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e23e108a-5368-4c32-8619-812481ed6820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:47.465897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1210/2409] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:47.470628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a488257-a79d-45b1-b393-43768256bb39 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.533769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a488257-a79d-45b1-b393-43768256bb39 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.538534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228249f-7099-436d-9ee2-08ee5a9ed9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.539196 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228249f-7099-436d-9ee2-08ee5a9ed9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.539484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228249f-7099-436d-9ee2-08ee5a9ed9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.539762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228249f-7099-436d-9ee2-08ee5a9ed9a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.540585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228249f-7099-436d-9ee2-08ee5a9ed9a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.542590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228249f-7099-436d-9ee2-08ee5a9ed9a7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.569285 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1200/2410] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:37:47 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:37:47.573182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e108a-5368-4c32-8619-812481ed6820 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.584089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e108a-5368-4c32-8619-812481ed6820 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.611134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:47.643137 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf38cd1-6ba6-442d-99e2-e2863847018e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.643804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf38cd1-6ba6-442d-99e2-e2863847018e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.644086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf38cd1-6ba6-442d-99e2-e2863847018e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.644361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf38cd1-6ba6-442d-99e2-e2863847018e admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.645081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf38cd1-6ba6-442d-99e2-e2863847018e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.647102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf38cd1-6ba6-442d-99e2-e2863847018e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.666391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1211/2411] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:37:47 2022] POST /identity/v3/users => generated 308 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:37:47.670400 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228249f-7099-436d-9ee2-08ee5a9ed9a7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.681454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228249f-7099-436d-9ee2-08ee5a9ed9a7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.686096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cb458-032e-437a-9a23-75862a00fce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.687432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb458-032e-437a-9a23-75862a00fce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.687727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb458-032e-437a-9a23-75862a00fce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.688025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb458-032e-437a-9a23-75862a00fce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.688989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cb458-032e-437a-9a23-75862a00fce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.690621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cb458-032e-437a-9a23-75862a00fce2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.701045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1201/2412] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:37:47.705716 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf38cd1-6ba6-442d-99e2-e2863847018e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.715512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf38cd1-6ba6-442d-99e2-e2863847018e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.721183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66b876-a751-4abc-93fa-80556e138209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.722096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66b876-a751-4abc-93fa-80556e138209 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.722380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66b876-a751-4abc-93fa-80556e138209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.722656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66b876-a751-4abc-93fa-80556e138209 admin admin] PATH_INFO: `/v3/projects/6bb989663ad24e9eb29f3c4658bdeb5f/users/0bf97b578dbe4ce28b2ff82057e26544/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.739287 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66b876-a751-4abc-93fa-80556e138209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb989663ad24e9eb29f3c4658bdeb5f, user_id=0bf97b578dbe4ce28b2ff82057e26544, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.741931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66b876-a751-4abc-93fa-80556e138209 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.754747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1212/2413] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:37:47 2022] PUT /identity/v3/projects/6bb989663ad24e9eb29f3c4658bdeb5f/users/0bf97b578dbe4ce28b2ff82057e26544/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:47.758781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cb458-032e-437a-9a23-75862a00fce2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.769321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cb458-032e-437a-9a23-75862a00fce2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.825259 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2469c-fd3d-4c71-b729-e80304145006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.825839 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2469c-fd3d-4c71-b729-e80304145006 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.826149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2469c-fd3d-4c71-b729-e80304145006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.826423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2469c-fd3d-4c71-b729-e80304145006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.827141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2469c-fd3d-4c71-b729-e80304145006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.829185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2469c-fd3d-4c71-b729-e80304145006 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.838695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1202/2414] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:37:47.842794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66b876-a751-4abc-93fa-80556e138209 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.853643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66b876-a751-4abc-93fa-80556e138209 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:47.858209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e5da4-c117-4e6b-af4d-3a07e613874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:47.860144 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e5da4-c117-4e6b-af4d-3a07e613874a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.860438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e5da4-c117-4e6b-af4d-3a07e613874a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.860715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e5da4-c117-4e6b-af4d-3a07e613874a admin admin] PATH_INFO: `/v3/projects/6bb989663ad24e9eb29f3c4658bdeb5f/users/0bf97b578dbe4ce28b2ff82057e26544/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.862656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e5da4-c117-4e6b-af4d-3a07e613874a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb989663ad24e9eb29f3c4658bdeb5f, user_id=0bf97b578dbe4ce28b2ff82057e26544, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:47.865249 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e5da4-c117-4e6b-af4d-3a07e613874a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:47.875951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1213/2415] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:37:47 2022] PUT /identity/v3/projects/6bb989663ad24e9eb29f3c4658bdeb5f/users/0bf97b578dbe4ce28b2ff82057e26544/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:47.886170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcebc3-970f-40b2-b1fd-8d32cc9d7a1d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:47.886467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcebc3-970f-40b2-b1fd-8d32cc9d7a1d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:47.886751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcebc3-970f-40b2-b1fd-8d32cc9d7a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:47.914019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c7bcebc3-970f-40b2-b1fd-8d32cc9d7a1d None None] MFA Rules not processed for user `0bf97b578dbe4ce28b2ff82057e26544`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:47.915281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c7bcebc3-970f-40b2-b1fd-8d32cc9d7a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:47.932041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:47.985496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1203/2416] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:47.996471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e5da4-c117-4e6b-af4d-3a07e613874a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:47.998216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c31e5da4-c117-4e6b-af4d-3a07e613874a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:48.065559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e5da4-c117-4e6b-af4d-3a07e613874a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:48.069220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08d4da-1aba-414c-96d7-42c82c6659a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:48.069873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08d4da-1aba-414c-96d7-42c82c6659a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:48.070197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08d4da-1aba-414c-96d7-42c82c6659a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:48.070574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08d4da-1aba-414c-96d7-42c82c6659a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:48.082226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08d4da-1aba-414c-96d7-42c82c6659a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:48.082357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08d4da-1aba-414c-96d7-42c82c6659a3 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:48.092313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1214/2417] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:37:47 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:37:48.247510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcebc3-970f-40b2-b1fd-8d32cc9d7a1d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:48.315602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcebc3-970f-40b2-b1fd-8d32cc9d7a1d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:48.320107 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:48.320984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:48.321266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:48.321542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] PATH_INFO: `/v3/users/ec7b14cd0ec84062bf7c235a54838e95` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:48.322760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec7b14cd0ec84062bf7c235a54838e95)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:48.324716 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:48.371208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec7b14cd0ec84062bf7c235a54838e95'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:48.371787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec7b14cd0ec84062bf7c235a54838e95'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:48.378999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec7b14cd0ec84062bf7c235a54838e95'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:48.391371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec7b14cd0ec84062bf7c235a54838e95'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:48.392009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec7b14cd0ec84062bf7c235a54838e95'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:48.406362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1204/2418] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:37:48 2022] DELETE /identity/v3/users/ec7b14cd0ec84062bf7c235a54838e95 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:48.411298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac70d-448f-49bf-b925-5b8ae7f967c4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:48.411941 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac70d-448f-49bf-b925-5b8ae7f967c4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:48.412273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac70d-448f-49bf-b925-5b8ae7f967c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:48.432703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-af2ac70d-448f-49bf-b925-5b8ae7f967c4 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:48.434265 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-af2ac70d-448f-49bf-b925-5b8ae7f967c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:48.494281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1215/2419] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:37:48.505095 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:48.506550 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:48.542974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:48.569327 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86166d67-e89f-4f19-a79c-732a4f3e8f42 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:48.573021 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8264a0-5af1-4f70-afc9-19213fcf3c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:48.573671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8264a0-5af1-4f70-afc9-19213fcf3c9b service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:48.573990 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8264a0-5af1-4f70-afc9-19213fcf3c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:48.574272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8264a0-5af1-4f70-afc9-19213fcf3c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:48.583985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8264a0-5af1-4f70-afc9-19213fcf3c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:48.585409 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8264a0-5af1-4f70-afc9-19213fcf3c9b service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:48.595003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1205/2420] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:37:48.865297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ac70d-448f-49bf-b925-5b8ae7f967c4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:48.866757 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-af2ac70d-448f-49bf-b925-5b8ae7f967c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:48.880074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ac70d-448f-49bf-b925-5b8ae7f967c4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:48.884614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:48.885483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:48.885763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:48.886075 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] PATH_INFO: `/v3/projects/0eaba88de9e349a7852a6c6a92650fc7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:48.887247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eaba88de9e349a7852a6c6a92650fc7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:48.889242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:48.911787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:48.940192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eaba88de9e349a7852a6c6a92650fc7', 'request_id': 'req-7770b331-f73d-4547-8e08-1cd60245d7ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:48.947360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eaba88de9e349a7852a6c6a92650fc7', 'request_id': 'req-7770b331-f73d-4547-8e08-1cd60245d7ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:48.953884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eaba88de9e349a7852a6c6a92650fc7', 'request_id': 'req-7770b331-f73d-4547-8e08-1cd60245d7ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:48.964126 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] The token cache is being invalidate because project 0eaba88de9e349a7852a6c6a92650fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:37:48.964413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:48.965050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:48.966630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1216/2421] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:37:48 2022] DELETE /identity/v3/projects/0eaba88de9e349a7852a6c6a92650fc7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:48.971242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8264a0-5af1-4f70-afc9-19213fcf3c9b service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:48.980794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8264a0-5af1-4f70-afc9-19213fcf3c9b service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:49.037913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1001d2-4097-4b8b-8c4d-6ed8fc8c120a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:49.038813 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1001d2-4097-4b8b-8c4d-6ed8fc8c120a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:49.039126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1001d2-4097-4b8b-8c4d-6ed8fc8c120a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:49.039499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1001d2-4097-4b8b-8c4d-6ed8fc8c120a admin admin] PATH_INFO: `/v3/users/0bf97b578dbe4ce28b2ff82057e26544` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:49.040853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1001d2-4097-4b8b-8c4d-6ed8fc8c120a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf97b578dbe4ce28b2ff82057e26544)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:49.042823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1001d2-4097-4b8b-8c4d-6ed8fc8c120a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:49.088605 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b1001d2-4097-4b8b-8c4d-6ed8fc8c120a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf97b578dbe4ce28b2ff82057e26544'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:49.089246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b1001d2-4097-4b8b-8c4d-6ed8fc8c120a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf97b578dbe4ce28b2ff82057e26544'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:49.096299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b1001d2-4097-4b8b-8c4d-6ed8fc8c120a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf97b578dbe4ce28b2ff82057e26544'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:49.107731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b1001d2-4097-4b8b-8c4d-6ed8fc8c120a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf97b578dbe4ce28b2ff82057e26544'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:49.108352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b1001d2-4097-4b8b-8c4d-6ed8fc8c120a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf97b578dbe4ce28b2ff82057e26544'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:49.122071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1206/2422] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:37:48 2022] DELETE /identity/v3/users/0bf97b578dbe4ce28b2ff82057e26544 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:49.407618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:49.409979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:49.422741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770b331-f73d-4547-8e08-1cd60245d7ab admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:49.489859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:49.490982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:49.491167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:49.491424 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] PATH_INFO: `/v3/projects/6bb989663ad24e9eb29f3c4658bdeb5f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:49.492627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb989663ad24e9eb29f3c4658bdeb5f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:49.494589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:49.545416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb989663ad24e9eb29f3c4658bdeb5f', 'request_id': 'req-546b0cd0-8510-4146-944a-71879d3b994c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:49.552897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb989663ad24e9eb29f3c4658bdeb5f', 'request_id': 'req-546b0cd0-8510-4146-944a-71879d3b994c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:49.559076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb989663ad24e9eb29f3c4658bdeb5f', 'request_id': 'req-546b0cd0-8510-4146-944a-71879d3b994c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:49.570414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] The token cache is being invalidate because project 6bb989663ad24e9eb29f3c4658bdeb5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:37:49.570700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:49.571330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-546b0cd0-8510-4146-944a-71879d3b994c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:49.572854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1217/2423] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:37:49 2022] DELETE /identity/v3/projects/6bb989663ad24e9eb29f3c4658bdeb5f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:49.578893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81287ba1-e10d-40e3-ae68-28642870aecf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:49.579349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81287ba1-e10d-40e3-ae68-28642870aecf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:49.579492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81287ba1-e10d-40e3-ae68-28642870aecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:49.597427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-81287ba1-e10d-40e3-ae68-28642870aecf None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:49.598760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-81287ba1-e10d-40e3-ae68-28642870aecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:49.646935 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:49.659602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1207/2424] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:49.668357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e7955-0127-4477-a6b8-c5844e7dca83 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:49.668650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e7955-0127-4477-a6b8-c5844e7dca83 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:49.668936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e7955-0127-4477-a6b8-c5844e7dca83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:49.688768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bb2e7955-0127-4477-a6b8-c5844e7dca83 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:49.690009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bb2e7955-0127-4477-a6b8-c5844e7dca83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:49.698246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1218/2425] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:49.702642 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81287ba1-e10d-40e3-ae68-28642870aecf None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:49.712672 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81287ba1-e10d-40e3-ae68-28642870aecf None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:49.717238 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed878a-094f-45bb-83cd-f0f58f942a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:49.717994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed878a-094f-45bb-83cd-f0f58f942a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:49.718419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed878a-094f-45bb-83cd-f0f58f942a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:49.718708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed878a-094f-45bb-83cd-f0f58f942a62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:49.719448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed878a-094f-45bb-83cd-f0f58f942a62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:37:49.719762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed878a-094f-45bb-83cd-f0f58f942a62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:49.721372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed878a-094f-45bb-83cd-f0f58f942a62 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:49.733340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1208/2426] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:37:49.739055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e7955-0127-4477-a6b8-c5844e7dca83 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:49.748392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e7955-0127-4477-a6b8-c5844e7dca83 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:49.753707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c44cd-10c3-4a0c-be0f-015de46ca0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:49.754460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c44cd-10c3-4a0c-be0f-015de46ca0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:49.754748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c44cd-10c3-4a0c-be0f-015de46ca0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:49.755028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c44cd-10c3-4a0c-be0f-015de46ca0a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:49.755741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c44cd-10c3-4a0c-be0f-015de46ca0a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:49.757701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c44cd-10c3-4a0c-be0f-015de46ca0a4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:49.783134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1219/2427] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:37:49 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:37:49.787297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed878a-094f-45bb-83cd-f0f58f942a62 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:49.798350 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed878a-094f-45bb-83cd-f0f58f942a62 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:49.856217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cc3a8-c415-4ef8-b3a6-14ff7884f5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:49.856889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc3a8-c415-4ef8-b3a6-14ff7884f5bf admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:49.857176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc3a8-c415-4ef8-b3a6-14ff7884f5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:49.857460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc3a8-c415-4ef8-b3a6-14ff7884f5bf admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:49.858176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cc3a8-c415-4ef8-b3a6-14ff7884f5bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:49.860624 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cc3a8-c415-4ef8-b3a6-14ff7884f5bf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:49.880984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1209/2428] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:37:49 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:37:49.885071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c44cd-10c3-4a0c-be0f-015de46ca0a4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:49.895636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c44cd-10c3-4a0c-be0f-015de46ca0a4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:49.900244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887e743-8d8b-4253-87aa-132f3a9ea8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:49.900901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887e743-8d8b-4253-87aa-132f3a9ea8ba admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:49.901186 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887e743-8d8b-4253-87aa-132f3a9ea8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:49.901729 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887e743-8d8b-4253-87aa-132f3a9ea8ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:49.902495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887e743-8d8b-4253-87aa-132f3a9ea8ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:49.904084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887e743-8d8b-4253-87aa-132f3a9ea8ba admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:49.906316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:49.917253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1220/2429] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:37:49.921587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cc3a8-c415-4ef8-b3a6-14ff7884f5bf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:49.931625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cc3a8-c415-4ef8-b3a6-14ff7884f5bf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:49.936317 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ec717-5840-47f2-a870-291454fbdb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:49.937628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ec717-5840-47f2-a870-291454fbdb16 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:49.937915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ec717-5840-47f2-a870-291454fbdb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:49.938227 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ec717-5840-47f2-a870-291454fbdb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:49.938965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ec717-5840-47f2-a870-291454fbdb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:49.940565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ec717-5840-47f2-a870-291454fbdb16 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:49.956364 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1210/2430] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:37:49.960518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887e743-8d8b-4253-87aa-132f3a9ea8ba admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:49.971170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887e743-8d8b-4253-87aa-132f3a9ea8ba admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:49.975745 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae1bce-6c29-492a-9a7b-36590a5160be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:49.976619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae1bce-6c29-492a-9a7b-36590a5160be admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:49.976898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae1bce-6c29-492a-9a7b-36590a5160be admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:49.977173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae1bce-6c29-492a-9a7b-36590a5160be admin admin] PATH_INFO: `/v3/projects/95f90829f3e6460280e58c085952beae/users/c93690fc7de24adbaf0e346385006f72/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:49.991993 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae1bce-6c29-492a-9a7b-36590a5160be admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f90829f3e6460280e58c085952beae, user_id=c93690fc7de24adbaf0e346385006f72, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:49.994793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae1bce-6c29-492a-9a7b-36590a5160be admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:50.007760 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1221/2431] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:37:49 2022] PUT /identity/v3/projects/95f90829f3e6460280e58c085952beae/users/c93690fc7de24adbaf0e346385006f72/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:37:50.018164 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a76627-08ad-40e1-a9b7-69b93c723d03 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:50.018463 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a76627-08ad-40e1-a9b7-69b93c723d03 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:50.018777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a76627-08ad-40e1-a9b7-69b93c723d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:50.046845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b0a76627-08ad-40e1-a9b7-69b93c723d03 None None] MFA Rules not processed for user `c93690fc7de24adbaf0e346385006f72`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:50.048023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b0a76627-08ad-40e1-a9b7-69b93c723d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:50.114362 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1211/2432] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:50.118860 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae1bce-6c29-492a-9a7b-36590a5160be admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:50.181352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae1bce-6c29-492a-9a7b-36590a5160be admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:50.186018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722d2f7-2efd-42c4-9999-d612a5aa9f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:50.186777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722d2f7-2efd-42c4-9999-d612a5aa9f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:50.187072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722d2f7-2efd-42c4-9999-d612a5aa9f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:50.187358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722d2f7-2efd-42c4-9999-d612a5aa9f6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:50.188054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722d2f7-2efd-42c4-9999-d612a5aa9f6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:50.190005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722d2f7-2efd-42c4-9999-d612a5aa9f6e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:50.215323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1222/2433] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:37:50 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:37:50.219741 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a76627-08ad-40e1-a9b7-69b93c723d03 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:50.230584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a76627-08ad-40e1-a9b7-69b93c723d03 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:50.281584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:50.293170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62f1ce-6121-405e-bf23-8f96eeef0480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:50.293853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62f1ce-6121-405e-bf23-8f96eeef0480 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:50.294199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62f1ce-6121-405e-bf23-8f96eeef0480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:50.294483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62f1ce-6121-405e-bf23-8f96eeef0480 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:50.295194 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62f1ce-6121-405e-bf23-8f96eeef0480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:50.297847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62f1ce-6121-405e-bf23-8f96eeef0480 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:50.316335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1212/2434] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:37:50 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:37:50.320381 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722d2f7-2efd-42c4-9999-d612a5aa9f6e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:50.330650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722d2f7-2efd-42c4-9999-d612a5aa9f6e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:50.335158 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fa7f6-3105-4c83-b8f5-33f2fb5c2864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:50.335811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fa7f6-3105-4c83-b8f5-33f2fb5c2864 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:50.336098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fa7f6-3105-4c83-b8f5-33f2fb5c2864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:50.336396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fa7f6-3105-4c83-b8f5-33f2fb5c2864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:50.337123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fa7f6-3105-4c83-b8f5-33f2fb5c2864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:50.338976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fa7f6-3105-4c83-b8f5-33f2fb5c2864 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:50.350212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1223/2435] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:37:50.354380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62f1ce-6121-405e-bf23-8f96eeef0480 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:50.365664 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62f1ce-6121-405e-bf23-8f96eeef0480 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:50.370338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3b6ca-8c13-4d73-a7b1-dc66f119dd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:50.371216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3b6ca-8c13-4d73-a7b1-dc66f119dd27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:50.371496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3b6ca-8c13-4d73-a7b1-dc66f119dd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:50.371778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3b6ca-8c13-4d73-a7b1-dc66f119dd27 admin admin] PATH_INFO: `/v3/projects/eae3c54a11fb4646b648e6d4a62d4f9b/users/cf4c71d859a341d6980e154222690004/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:50.387023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3b6ca-8c13-4d73-a7b1-dc66f119dd27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae3c54a11fb4646b648e6d4a62d4f9b, user_id=cf4c71d859a341d6980e154222690004, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:50.389670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3b6ca-8c13-4d73-a7b1-dc66f119dd27 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:50.403111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1213/2436] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:37:50 2022] PUT /identity/v3/projects/eae3c54a11fb4646b648e6d4a62d4f9b/users/cf4c71d859a341d6980e154222690004/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:50.407758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fa7f6-3105-4c83-b8f5-33f2fb5c2864 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:50.417735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fa7f6-3105-4c83-b8f5-33f2fb5c2864 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:50.474545 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6f2b1-42fc-459d-800e-b402dbcf2696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:50.475219 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6f2b1-42fc-459d-800e-b402dbcf2696 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:50.475504 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6f2b1-42fc-459d-800e-b402dbcf2696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:50.475781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6f2b1-42fc-459d-800e-b402dbcf2696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:50.476505 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6f2b1-42fc-459d-800e-b402dbcf2696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:50.478125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6f2b1-42fc-459d-800e-b402dbcf2696 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:50.488159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1224/2437] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:37:50.492199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3b6ca-8c13-4d73-a7b1-dc66f119dd27 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:50.502470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3b6ca-8c13-4d73-a7b1-dc66f119dd27 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:50.507057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ed4f2-6618-4448-9c05-63e43944174a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:50.507934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ed4f2-6618-4448-9c05-63e43944174a admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:50.508216 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ed4f2-6618-4448-9c05-63e43944174a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:50.508492 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ed4f2-6618-4448-9c05-63e43944174a admin admin] PATH_INFO: `/v3/projects/eae3c54a11fb4646b648e6d4a62d4f9b/users/cf4c71d859a341d6980e154222690004/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:50.510535 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ed4f2-6618-4448-9c05-63e43944174a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae3c54a11fb4646b648e6d4a62d4f9b, user_id=cf4c71d859a341d6980e154222690004, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:50.513162 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ed4f2-6618-4448-9c05-63e43944174a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:50.524210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1214/2438] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:37:50 2022] PUT /identity/v3/projects/eae3c54a11fb4646b648e6d4a62d4f9b/users/cf4c71d859a341d6980e154222690004/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:50.536216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd17f6-582c-469a-b8b0-9640b784a907 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:50.536514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd17f6-582c-469a-b8b0-9640b784a907 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:50.536806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd17f6-582c-469a-b8b0-9640b784a907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:50.570730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c0cd17f6-582c-469a-b8b0-9640b784a907 None None] MFA Rules not processed for user `cf4c71d859a341d6980e154222690004`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:50.570730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c0cd17f6-582c-469a-b8b0-9640b784a907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:50.602867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:50.610678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ed4f2-6618-4448-9c05-63e43944174a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:50.612860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-813ed4f2-6618-4448-9c05-63e43944174a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:50.651675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1225/2439] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:50.663473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd17f6-582c-469a-b8b0-9640b784a907 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:50.665307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c0cd17f6-582c-469a-b8b0-9640b784a907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:50.693084 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ed4f2-6618-4448-9c05-63e43944174a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:50.696945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b6cb7-c522-4fc4-8fbf-2fcc2f5ccd2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:50.697597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b6cb7-c522-4fc4-8fbf-2fcc2f5ccd2c service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:50.697886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b6cb7-c522-4fc4-8fbf-2fcc2f5ccd2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:50.698204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b6cb7-c522-4fc4-8fbf-2fcc2f5ccd2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:50.700043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd4b6cb7-c522-4fc4-8fbf-2fcc2f5ccd2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:50.723986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd17f6-582c-469a-b8b0-9640b784a907 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:50.727703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6215c4-b8ad-4e51-868c-89f98c9e3343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:50.728357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6215c4-b8ad-4e51-868c-89f98c9e3343 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:50.728645 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6215c4-b8ad-4e51-868c-89f98c9e3343 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:50.728923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6215c4-b8ad-4e51-868c-89f98c9e3343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:50.738709 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6215c4-b8ad-4e51-868c-89f98c9e3343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:50.740058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6215c4-b8ad-4e51-868c-89f98c9e3343 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:50.750548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1226/2440] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:37:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:37:50.773682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b6cb7-c522-4fc4-8fbf-2fcc2f5ccd2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:50.775163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b6cb7-c522-4fc4-8fbf-2fcc2f5ccd2c service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:50.790143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1215/2441] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:37:50.822970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6215c4-b8ad-4e51-868c-89f98c9e3343 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:50.832835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6215c4-b8ad-4e51-868c-89f98c9e3343 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:50.842855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849084ac-3888-4a30-8dd5-c63b9e7bb55e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:50.843687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849084ac-3888-4a30-8dd5-c63b9e7bb55e service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:50.844047 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849084ac-3888-4a30-8dd5-c63b9e7bb55e service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:50.844404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849084ac-3888-4a30-8dd5-c63b9e7bb55e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:50.907357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849084ac-3888-4a30-8dd5-c63b9e7bb55e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:50.908952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849084ac-3888-4a30-8dd5-c63b9e7bb55e service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:50.920148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1227/2442] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:37:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:37:53.900522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b6cb7-c522-4fc4-8fbf-2fcc2f5ccd2c service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:53.902282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd4b6cb7-c522-4fc4-8fbf-2fcc2f5ccd2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:53.970081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:53.974747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849084ac-3888-4a30-8dd5-c63b9e7bb55e service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:53.976427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-849084ac-3888-4a30-8dd5-c63b9e7bb55e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:53.994517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b6cb7-c522-4fc4-8fbf-2fcc2f5ccd2c service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:53.996558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849084ac-3888-4a30-8dd5-c63b9e7bb55e service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:54.001146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb8ae9-5a07-4119-8b6b-b936378919c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da2d85eb18049f196aeefc51d3f1e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:54.001821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb8ae9-5a07-4119-8b6b-b936378919c6 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.002138 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb8ae9-5a07-4119-8b6b-b936378919c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.002418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb8ae9-5a07-4119-8b6b-b936378919c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.003353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50296775-70db-43f7-a1bd-5f00e03952a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:54.004359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50296775-70db-43f7-a1bd-5f00e03952a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.004491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d1bb8ae9-5a07-4119-8b6b-b936378919c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:54.004827 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50296775-70db-43f7-a1bd-5f00e03952a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.005647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50296775-70db-43f7-a1bd-5f00e03952a7 admin admin] PATH_INFO: `/v3/users/5c54ac39faa84c7db5b9b63360216d7b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.007242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50296775-70db-43f7-a1bd-5f00e03952a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c54ac39faa84c7db5b9b63360216d7b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:54.009490 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50296775-70db-43f7-a1bd-5f00e03952a7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:54.017395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb8ae9-5a07-4119-8b6b-b936378919c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:54.019185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb8ae9-5a07-4119-8b6b-b936378919c6 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:54.031091 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1216/2443] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:37:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:37:54.079187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-50296775-70db-43f7-a1bd-5f00e03952a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c54ac39faa84c7db5b9b63360216d7b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:54.091291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-50296775-70db-43f7-a1bd-5f00e03952a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c54ac39faa84c7db5b9b63360216d7b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:54.092488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:54.100486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-50296775-70db-43f7-a1bd-5f00e03952a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c54ac39faa84c7db5b9b63360216d7b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:54.101184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-50296775-70db-43f7-a1bd-5f00e03952a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c54ac39faa84c7db5b9b63360216d7b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:54.101815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-50296775-70db-43f7-a1bd-5f00e03952a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c54ac39faa84c7db5b9b63360216d7b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:54.115221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1228/2444] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:37:53 2022] DELETE /identity/v3/users/5c54ac39faa84c7db5b9b63360216d7b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:37:54.286558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb8ae9-5a07-4119-8b6b-b936378919c6 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:54.299681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb8ae9-5a07-4119-8b6b-b936378919c6 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:54.360750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:54.361770 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.362190 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.362649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] PATH_INFO: `/v3/projects/f1b4bcbfdd4a42e5a542dbf019166beb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.364165 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b4bcbfdd4a42e5a542dbf019166beb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:54.366198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:54.411406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b4bcbfdd4a42e5a542dbf019166beb', 'request_id': 'req-c4f41aff-3c41-492e-b088-1ec23f9070d6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:54.416822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b4bcbfdd4a42e5a542dbf019166beb', 'request_id': 'req-c4f41aff-3c41-492e-b088-1ec23f9070d6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:54.423369 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b4bcbfdd4a42e5a542dbf019166beb', 'request_id': 'req-c4f41aff-3c41-492e-b088-1ec23f9070d6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:54.435171 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] The token cache is being invalidate because project f1b4bcbfdd4a42e5a542dbf019166beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:37:54.435462 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:54.436077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4f41aff-3c41-492e-b088-1ec23f9070d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:37:54.438553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1217/2445] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:37:54 2022] DELETE /identity/v3/projects/f1b4bcbfdd4a42e5a542dbf019166beb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:54.443720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf07e8-4a9a-4d23-8f77-be658d602e11 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.444015 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf07e8-4a9a-4d23-8f77-be658d602e11 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.444295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf07e8-4a9a-4d23-8f77-be658d602e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.462484 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ccbf07e8-4a9a-4d23-8f77-be658d602e11 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:54.463668 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ccbf07e8-4a9a-4d23-8f77-be658d602e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:54.521978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1229/2446] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:54.531102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e090d8-c23d-4a14-aeb7-fce17f17a342 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.531495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e090d8-c23d-4a14-aeb7-fce17f17a342 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.531887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e090d8-c23d-4a14-aeb7-fce17f17a342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.549882 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d9e090d8-c23d-4a14-aeb7-fce17f17a342 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:54.551108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d9e090d8-c23d-4a14-aeb7-fce17f17a342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:54.558986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1218/2447] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:54.563697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf07e8-4a9a-4d23-8f77-be658d602e11 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:54.573435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf07e8-4a9a-4d23-8f77-be658d602e11 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:54.578985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc05e0-e012-4042-b518-b67d142918b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:54.579656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc05e0-e012-4042-b518-b67d142918b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.579943 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc05e0-e012-4042-b518-b67d142918b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.580221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc05e0-e012-4042-b518-b67d142918b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.580960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc05e0-e012-4042-b518-b67d142918b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:37:54.581270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc05e0-e012-4042-b518-b67d142918b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:54.582912 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc05e0-e012-4042-b518-b67d142918b5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:54.594688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1230/2448] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:37:54.599305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e090d8-c23d-4a14-aeb7-fce17f17a342 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:54.608903 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e090d8-c23d-4a14-aeb7-fce17f17a342 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:54.613723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ff8e0-b7e8-4c68-8714-1774aafbb547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:54.614421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ff8e0-b7e8-4c68-8714-1774aafbb547 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.614704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ff8e0-b7e8-4c68-8714-1774aafbb547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.614982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ff8e0-b7e8-4c68-8714-1774aafbb547 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.615686 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ff8e0-b7e8-4c68-8714-1774aafbb547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:54.617593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ff8e0-b7e8-4c68-8714-1774aafbb547 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:54.647421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1219/2449] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:37:54 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:37:54.648495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc05e0-e012-4042-b518-b67d142918b5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:54.658686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc05e0-e012-4042-b518-b67d142918b5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:54.714016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563c4bc-503b-4600-9bce-b2b06481c74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:54.714680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563c4bc-503b-4600-9bce-b2b06481c74a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.714980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563c4bc-503b-4600-9bce-b2b06481c74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.715247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563c4bc-503b-4600-9bce-b2b06481c74a admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.715937 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563c4bc-503b-4600-9bce-b2b06481c74a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:54.717853 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563c4bc-503b-4600-9bce-b2b06481c74a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:54.737919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1231/2450] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:37:54 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:37:54.741797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ff8e0-b7e8-4c68-8714-1774aafbb547 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:54.752874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ff8e0-b7e8-4c68-8714-1774aafbb547 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:54.757485 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeeef7a-1e5b-45a7-913d-2803a6b4b6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:54.758182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeef7a-1e5b-45a7-913d-2803a6b4b6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.758469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeef7a-1e5b-45a7-913d-2803a6b4b6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.758750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeef7a-1e5b-45a7-913d-2803a6b4b6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.759481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeeef7a-1e5b-45a7-913d-2803a6b4b6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:54.761112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeeef7a-1e5b-45a7-913d-2803a6b4b6b8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:54.772673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1220/2451] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:37:54.776848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563c4bc-503b-4600-9bce-b2b06481c74a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:54.787048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563c4bc-503b-4600-9bce-b2b06481c74a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:54.791529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70eb75-1f20-4eb5-9fc1-26f54fb293f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:54.792183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70eb75-1f20-4eb5-9fc1-26f54fb293f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.792467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70eb75-1f20-4eb5-9fc1-26f54fb293f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.792746 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70eb75-1f20-4eb5-9fc1-26f54fb293f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.793472 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70eb75-1f20-4eb5-9fc1-26f54fb293f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:54.795087 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70eb75-1f20-4eb5-9fc1-26f54fb293f4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:54.805443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1232/2452] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:37:54.809256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeeef7a-1e5b-45a7-913d-2803a6b4b6b8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:54.820156 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeeef7a-1e5b-45a7-913d-2803a6b4b6b8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:54.825113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add520bf-d8fb-4fc4-9764-d6eebde1a00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:54.826043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add520bf-d8fb-4fc4-9764-d6eebde1a00c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.826329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add520bf-d8fb-4fc4-9764-d6eebde1a00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.826607 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add520bf-d8fb-4fc4-9764-d6eebde1a00c admin admin] PATH_INFO: `/v3/projects/5466cef7e1e842038f95082be46e9447/users/f6b98dd545eb4ebb9c819f82103fae52/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.842176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add520bf-d8fb-4fc4-9764-d6eebde1a00c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5466cef7e1e842038f95082be46e9447, user_id=f6b98dd545eb4ebb9c819f82103fae52, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:54.844893 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add520bf-d8fb-4fc4-9764-d6eebde1a00c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:54.857733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1221/2453] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:37:54 2022] PUT /identity/v3/projects/5466cef7e1e842038f95082be46e9447/users/f6b98dd545eb4ebb9c819f82103fae52/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:37:54.868526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d410e6-4c13-4bee-8878-32708692c909 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:54.868822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d410e6-4c13-4bee-8878-32708692c909 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:54.869100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d410e6-4c13-4bee-8878-32708692c909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:54.896939 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a5d410e6-4c13-4bee-8878-32708692c909 None None] MFA Rules not processed for user `f6b98dd545eb4ebb9c819f82103fae52`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:37:54.898178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a5d410e6-4c13-4bee-8878-32708692c909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:54.944070 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:54.972019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1233/2454] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:37:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:37:54.982997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add520bf-d8fb-4fc4-9764-d6eebde1a00c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:37:54.985649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-add520bf-d8fb-4fc4-9764-d6eebde1a00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:37:54.996793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:37:55.063343 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add520bf-d8fb-4fc4-9764-d6eebde1a00c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:37:55.068866 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd5ee-5383-4143-9ba7-c6e358c0b44d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:37:55.069515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd5ee-5383-4143-9ba7-c6e358c0b44d service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:37:55.069797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd5ee-5383-4143-9ba7-c6e358c0b44d service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:37:55.070112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd5ee-5383-4143-9ba7-c6e358c0b44d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:37:55.078308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfd5ee-5383-4143-9ba7-c6e358c0b44d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:37:55.079917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfd5ee-5383-4143-9ba7-c6e358c0b44d service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:37:55.088628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1222/2455] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:37:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:00.500684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d410e6-4c13-4bee-8878-32708692c909 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:00.502119 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a5d410e6-4c13-4bee-8878-32708692c909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:00.558095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d410e6-4c13-4bee-8878-32708692c909 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:00.561651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c855c-14fe-46ab-927d-22e0f5c33482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:00.562979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c855c-14fe-46ab-927d-22e0f5c33482 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:00.563263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c855c-14fe-46ab-927d-22e0f5c33482 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:00.563544 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c855c-14fe-46ab-927d-22e0f5c33482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:00.565303 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2d9c855c-14fe-46ab-927d-22e0f5c33482 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:00.623345 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c855c-14fe-46ab-927d-22e0f5c33482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:00.625406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c855c-14fe-46ab-927d-22e0f5c33482 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:00.634437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1234/2456] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:01.974515 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd5ee-5383-4143-9ba7-c6e358c0b44d service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:01.975917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-46bfd5ee-5383-4143-9ba7-c6e358c0b44d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:02.040696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd5ee-5383-4143-9ba7-c6e358c0b44d service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:02.045284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:02.046232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:02.046554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:02.046554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] PATH_INFO: `/v3/users/c93690fc7de24adbaf0e346385006f72` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:02.047996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93690fc7de24adbaf0e346385006f72)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:02.049914 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:02.097175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93690fc7de24adbaf0e346385006f72'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.097826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93690fc7de24adbaf0e346385006f72'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.104746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93690fc7de24adbaf0e346385006f72'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.122098 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93690fc7de24adbaf0e346385006f72'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.123071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93690fc7de24adbaf0e346385006f72'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.137834 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1223/2457] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:01 2022] DELETE /identity/v3/users/c93690fc7de24adbaf0e346385006f72 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:02.143214 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b492cd4-98e8-49b2-82b6-1b8b5b89bdbe None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:02.143688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b492cd4-98e8-49b2-82b6-1b8b5b89bdbe None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:02.143817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b492cd4-98e8-49b2-82b6-1b8b5b89bdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:02.161919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3b492cd4-98e8-49b2-82b6-1b8b5b89bdbe None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:02.163131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3b492cd4-98e8-49b2-82b6-1b8b5b89bdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:02.222333 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1235/2458] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:02.232074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:02.233479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:02.248817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:02.295279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e910aae-6c96-4591-869c-b38ce4586ce0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:02.300059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48cd2c-a654-424f-afc6-499b23e7c3a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:02.300685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48cd2c-a654-424f-afc6-499b23e7c3a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:02.300968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48cd2c-a654-424f-afc6-499b23e7c3a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:02.301247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48cd2c-a654-424f-afc6-499b23e7c3a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:02.310323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48cd2c-a654-424f-afc6-499b23e7c3a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:02.311735 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48cd2c-a654-424f-afc6-499b23e7c3a2 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:02.323102 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1224/2459] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:02.594035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b492cd4-98e8-49b2-82b6-1b8b5b89bdbe None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:02.595646 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3b492cd4-98e8-49b2-82b6-1b8b5b89bdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:02.607916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b492cd4-98e8-49b2-82b6-1b8b5b89bdbe None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:02.612608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:02.613482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:02.613771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:02.614084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] PATH_INFO: `/v3/projects/95f90829f3e6460280e58c085952beae` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:02.615288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f90829f3e6460280e58c085952beae)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:02.617334 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:02.650994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:02.667920 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f90829f3e6460280e58c085952beae', 'request_id': 'req-6af7d90c-c305-47c9-9a11-c0a29a9d1402'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.675445 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f90829f3e6460280e58c085952beae', 'request_id': 'req-6af7d90c-c305-47c9-9a11-c0a29a9d1402'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.681801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f90829f3e6460280e58c085952beae', 'request_id': 'req-6af7d90c-c305-47c9-9a11-c0a29a9d1402'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.692404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] The token cache is being invalidate because project 95f90829f3e6460280e58c085952beae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:02.692691 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.693323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.694890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1236/2460] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:02 2022] DELETE /identity/v3/projects/95f90829f3e6460280e58c085952beae => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:02.698845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48cd2c-a654-424f-afc6-499b23e7c3a2 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:02.709415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48cd2c-a654-424f-afc6-499b23e7c3a2 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:02.768260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee85e4a-c47f-4f10-ad91-c53193cb3a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:02.769054 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee85e4a-c47f-4f10-ad91-c53193cb3a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:02.769336 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee85e4a-c47f-4f10-ad91-c53193cb3a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:02.769611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee85e4a-c47f-4f10-ad91-c53193cb3a7a admin admin] PATH_INFO: `/v3/users/cf4c71d859a341d6980e154222690004` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:02.770819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee85e4a-c47f-4f10-ad91-c53193cb3a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf4c71d859a341d6980e154222690004)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:02.772767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee85e4a-c47f-4f10-ad91-c53193cb3a7a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:02.820428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-aee85e4a-c47f-4f10-ad91-c53193cb3a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf4c71d859a341d6980e154222690004'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.821072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-aee85e4a-c47f-4f10-ad91-c53193cb3a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf4c71d859a341d6980e154222690004'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.828644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-aee85e4a-c47f-4f10-ad91-c53193cb3a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf4c71d859a341d6980e154222690004'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.840990 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-aee85e4a-c47f-4f10-ad91-c53193cb3a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf4c71d859a341d6980e154222690004'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.841654 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-aee85e4a-c47f-4f10-ad91-c53193cb3a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf4c71d859a341d6980e154222690004'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:02.855963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1225/2461] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:02 2022] DELETE /identity/v3/users/cf4c71d859a341d6980e154222690004 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:03.133086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:03.134539 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:03.148171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7d90c-c305-47c9-9a11-c0a29a9d1402 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:03.216598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:03.217464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:03.217742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:03.218046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] PATH_INFO: `/v3/projects/eae3c54a11fb4646b648e6d4a62d4f9b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:03.219233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae3c54a11fb4646b648e6d4a62d4f9b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:03.221125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:03.266537 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae3c54a11fb4646b648e6d4a62d4f9b', 'request_id': 'req-acd8b13e-fd1f-49cf-be24-2e12b229f014'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:03.273751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae3c54a11fb4646b648e6d4a62d4f9b', 'request_id': 'req-acd8b13e-fd1f-49cf-be24-2e12b229f014'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:03.280224 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae3c54a11fb4646b648e6d4a62d4f9b', 'request_id': 'req-acd8b13e-fd1f-49cf-be24-2e12b229f014'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:03.290735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] The token cache is being invalidate because project eae3c54a11fb4646b648e6d4a62d4f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:03.291054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:03.291700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-acd8b13e-fd1f-49cf-be24-2e12b229f014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:03.293232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1237/2462] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:03 2022] DELETE /identity/v3/projects/eae3c54a11fb4646b648e6d4a62d4f9b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:03.298926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4058dd-2834-4cf9-be48-8404923897d8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:03.299402 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4058dd-2834-4cf9-be48-8404923897d8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:03.299509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4058dd-2834-4cf9-be48-8404923897d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:03.318460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ac4058dd-2834-4cf9-be48-8404923897d8 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:03.319596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ac4058dd-2834-4cf9-be48-8404923897d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:03.349009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:03.380738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1226/2463] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:03.388541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9bc2c-6773-468b-93fc-aa3cbc80361a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:03.388839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9bc2c-6773-468b-93fc-aa3cbc80361a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:03.389143 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9bc2c-6773-468b-93fc-aa3cbc80361a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:03.407368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-68f9bc2c-6773-468b-93fc-aa3cbc80361a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:03.408561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-68f9bc2c-6773-468b-93fc-aa3cbc80361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:03.416575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1238/2464] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:03.420633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4058dd-2834-4cf9-be48-8404923897d8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:03.430708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4058dd-2834-4cf9-be48-8404923897d8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:03.435284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12a7a5-2138-4c10-af90-38cc0e65d339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:03.435936 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12a7a5-2138-4c10-af90-38cc0e65d339 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:03.436218 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12a7a5-2138-4c10-af90-38cc0e65d339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:03.436497 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12a7a5-2138-4c10-af90-38cc0e65d339 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:03.437925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12a7a5-2138-4c10-af90-38cc0e65d339 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:03.438269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12a7a5-2138-4c10-af90-38cc0e65d339 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:03.439866 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12a7a5-2138-4c10-af90-38cc0e65d339 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:03.451285 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1227/2465] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:03.455516 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9bc2c-6773-468b-93fc-aa3cbc80361a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:03.464673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9bc2c-6773-468b-93fc-aa3cbc80361a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:03.469882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6fa23-8f01-46b3-b39f-f46e2712dba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:03.470571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6fa23-8f01-46b3-b39f-f46e2712dba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:03.470861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6fa23-8f01-46b3-b39f-f46e2712dba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:03.471144 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6fa23-8f01-46b3-b39f-f46e2712dba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:03.471835 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6fa23-8f01-46b3-b39f-f46e2712dba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:03.473739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6fa23-8f01-46b3-b39f-f46e2712dba3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:03.498934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1239/2466] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:03 2022] POST /identity/v3/projects => generated 401 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:03.503225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12a7a5-2138-4c10-af90-38cc0e65d339 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:03.513103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12a7a5-2138-4c10-af90-38cc0e65d339 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:03.569619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60db066-a891-4854-bb7c-cbe11987d5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:03.570258 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60db066-a891-4854-bb7c-cbe11987d5ac admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:03.570552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60db066-a891-4854-bb7c-cbe11987d5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:03.570831 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60db066-a891-4854-bb7c-cbe11987d5ac admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:03.571533 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60db066-a891-4854-bb7c-cbe11987d5ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:03.573449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60db066-a891-4854-bb7c-cbe11987d5ac admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:03.593424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1228/2467] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:03 2022] POST /identity/v3/users => generated 301 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:03.597518 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6fa23-8f01-46b3-b39f-f46e2712dba3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:03.607918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6fa23-8f01-46b3-b39f-f46e2712dba3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:03.612692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24313d0b-bf20-49c2-964a-307591a35530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:03.613346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313d0b-bf20-49c2-964a-307591a35530 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:03.613629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313d0b-bf20-49c2-964a-307591a35530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:03.613905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313d0b-bf20-49c2-964a-307591a35530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:03.614665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24313d0b-bf20-49c2-964a-307591a35530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:03.616284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24313d0b-bf20-49c2-964a-307591a35530 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:03.627269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1240/2468] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:03.630951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60db066-a891-4854-bb7c-cbe11987d5ac admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:03.641426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60db066-a891-4854-bb7c-cbe11987d5ac admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:03.645930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab171e0-8c35-4610-bd81-81a92e485dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:03.646600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab171e0-8c35-4610-bd81-81a92e485dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:03.646887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab171e0-8c35-4610-bd81-81a92e485dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:03.647167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab171e0-8c35-4610-bd81-81a92e485dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:03.647891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab171e0-8c35-4610-bd81-81a92e485dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:03.649457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab171e0-8c35-4610-bd81-81a92e485dd2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:03.659665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1229/2469] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:03.663528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24313d0b-bf20-49c2-964a-307591a35530 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:03.673848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24313d0b-bf20-49c2-964a-307591a35530 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:03.684826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d50c6-0f43-4b8a-9bf0-406285509185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:03.685739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d50c6-0f43-4b8a-9bf0-406285509185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:03.686040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d50c6-0f43-4b8a-9bf0-406285509185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:03.686323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d50c6-0f43-4b8a-9bf0-406285509185 admin admin] PATH_INFO: `/v3/projects/4a95e572f9b14e2a84e6c4a4204277d0/users/f1c6347473ab4ceeae5dcbcb707b5110/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:03.689239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:03.705026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d50c6-0f43-4b8a-9bf0-406285509185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a95e572f9b14e2a84e6c4a4204277d0, user_id=f1c6347473ab4ceeae5dcbcb707b5110, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:03.707726 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d50c6-0f43-4b8a-9bf0-406285509185 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:03.721356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1241/2470] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:03 2022] PUT /identity/v3/projects/4a95e572f9b14e2a84e6c4a4204277d0/users/f1c6347473ab4ceeae5dcbcb707b5110/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:03.731731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef287a-da5b-465d-958d-038a45b6f1fe None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:03.732025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef287a-da5b-465d-958d-038a45b6f1fe None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:03.732311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef287a-da5b-465d-958d-038a45b6f1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:03.759622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-40ef287a-da5b-465d-958d-038a45b6f1fe None None] MFA Rules not processed for user `f1c6347473ab4ceeae5dcbcb707b5110`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:03.760864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-40ef287a-da5b-465d-958d-038a45b6f1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:03.825409 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1230/2471] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:03.830471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d50c6-0f43-4b8a-9bf0-406285509185 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:03.895538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d50c6-0f43-4b8a-9bf0-406285509185 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:03.900170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480106d-6dee-4f64-93a5-ed87ecba8e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:03.900841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480106d-6dee-4f64-93a5-ed87ecba8e47 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:03.901129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480106d-6dee-4f64-93a5-ed87ecba8e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:03.901410 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480106d-6dee-4f64-93a5-ed87ecba8e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:03.902457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480106d-6dee-4f64-93a5-ed87ecba8e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:03.904373 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480106d-6dee-4f64-93a5-ed87ecba8e47 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:03.931519 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1242/2472] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:03 2022] POST /identity/v3/projects => generated 399 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:03.935486 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef287a-da5b-465d-958d-038a45b6f1fe None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:03.946891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef287a-da5b-465d-958d-038a45b6f1fe None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:03.976335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:04.007608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ac805-761c-4276-bcbf-8bce3c78c8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:04.008282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ac805-761c-4276-bcbf-8bce3c78c8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:04.008563 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ac805-761c-4276-bcbf-8bce3c78c8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:04.008842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ac805-761c-4276-bcbf-8bce3c78c8b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:04.009559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ac805-761c-4276-bcbf-8bce3c78c8b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:04.011520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ac805-761c-4276-bcbf-8bce3c78c8b7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:04.035498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1231/2473] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:03 2022] POST /identity/v3/users => generated 306 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:04.036626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480106d-6dee-4f64-93a5-ed87ecba8e47 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:04.049247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480106d-6dee-4f64-93a5-ed87ecba8e47 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:04.054830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33b87c-f543-48a4-84a7-9d8e68fcf0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:04.055520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33b87c-f543-48a4-84a7-9d8e68fcf0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:04.055806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33b87c-f543-48a4-84a7-9d8e68fcf0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:04.056090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33b87c-f543-48a4-84a7-9d8e68fcf0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:04.057198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33b87c-f543-48a4-84a7-9d8e68fcf0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:04.058881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33b87c-f543-48a4-84a7-9d8e68fcf0a9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:04.071885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1243/2474] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:38:04.076146 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ac805-761c-4276-bcbf-8bce3c78c8b7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:04.086222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ac805-761c-4276-bcbf-8bce3c78c8b7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:04.091149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb649e2-df8a-4149-be30-a003239c32bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:04.092208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb649e2-df8a-4149-be30-a003239c32bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:04.092460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb649e2-df8a-4149-be30-a003239c32bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:04.092808 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb649e2-df8a-4149-be30-a003239c32bc admin admin] PATH_INFO: `/v3/projects/04b11ec86a254eecbff23c0efc50db25/users/f342354e01764850bdb169731851f432/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:04.108198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb649e2-df8a-4149-be30-a003239c32bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b11ec86a254eecbff23c0efc50db25, user_id=f342354e01764850bdb169731851f432, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:04.111704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb649e2-df8a-4149-be30-a003239c32bc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:04.126567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1232/2475] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:04 2022] PUT /identity/v3/projects/04b11ec86a254eecbff23c0efc50db25/users/f342354e01764850bdb169731851f432/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:04.130736 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33b87c-f543-48a4-84a7-9d8e68fcf0a9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:04.141217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33b87c-f543-48a4-84a7-9d8e68fcf0a9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:04.198601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e61fc-e43e-4ac2-b3a4-b05d8e4438a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:04.199340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e61fc-e43e-4ac2-b3a4-b05d8e4438a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:04.199634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e61fc-e43e-4ac2-b3a4-b05d8e4438a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:04.199919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e61fc-e43e-4ac2-b3a4-b05d8e4438a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:04.200660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e61fc-e43e-4ac2-b3a4-b05d8e4438a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:04.202307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e61fc-e43e-4ac2-b3a4-b05d8e4438a7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:04.213169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1244/2476] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:04.217405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb649e2-df8a-4149-be30-a003239c32bc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:04.233386 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb649e2-df8a-4149-be30-a003239c32bc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:04.238331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17686f52-e710-4539-87f3-e71057127541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:04.239307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686f52-e710-4539-87f3-e71057127541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:04.239657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686f52-e710-4539-87f3-e71057127541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:04.240002 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686f52-e710-4539-87f3-e71057127541 admin admin] PATH_INFO: `/v3/projects/04b11ec86a254eecbff23c0efc50db25/users/f342354e01764850bdb169731851f432/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:04.242175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17686f52-e710-4539-87f3-e71057127541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b11ec86a254eecbff23c0efc50db25, user_id=f342354e01764850bdb169731851f432, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:04.244804 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17686f52-e710-4539-87f3-e71057127541 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:04.255858 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1233/2477] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:04 2022] PUT /identity/v3/projects/04b11ec86a254eecbff23c0efc50db25/users/f342354e01764850bdb169731851f432/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:04.267054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d587d-c251-4b34-8e97-01265af31e45 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:04.267357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d587d-c251-4b34-8e97-01265af31e45 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:04.267641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d587d-c251-4b34-8e97-01265af31e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:04.303432 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-893d587d-c251-4b34-8e97-01265af31e45 None None] MFA Rules not processed for user `f342354e01764850bdb169731851f432`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:04.304651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-893d587d-c251-4b34-8e97-01265af31e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:04.350933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:04.380269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1245/2478] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:04 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:04.391112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17686f52-e710-4539-87f3-e71057127541 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:04.392832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-17686f52-e710-4539-87f3-e71057127541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:04.450244 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17686f52-e710-4539-87f3-e71057127541 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:04.454003 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4483e-1278-41ab-9093-fbd5afc69613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:04.454645 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4483e-1278-41ab-9093-fbd5afc69613 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:04.454930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4483e-1278-41ab-9093-fbd5afc69613 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:04.455206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4483e-1278-41ab-9093-fbd5afc69613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:04.464141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4483e-1278-41ab-9093-fbd5afc69613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:04.465566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4483e-1278-41ab-9093-fbd5afc69613 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:04.475540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1234/2479] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:04 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:04.549902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d587d-c251-4b34-8e97-01265af31e45 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:04.559593 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d587d-c251-4b34-8e97-01265af31e45 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:04.563667 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fc8cd-bd4b-4405-8394-e43a3c272196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:04.564313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fc8cd-bd4b-4405-8394-e43a3c272196 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:04.564597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fc8cd-bd4b-4405-8394-e43a3c272196 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:04.564875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fc8cd-bd4b-4405-8394-e43a3c272196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:04.627735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fc8cd-bd4b-4405-8394-e43a3c272196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:04.629126 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fc8cd-bd4b-4405-8394-e43a3c272196 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:04.638873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1246/2480] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:05.506667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4483e-1278-41ab-9093-fbd5afc69613 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:05.506667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6bf4483e-1278-41ab-9093-fbd5afc69613 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:05.571495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4483e-1278-41ab-9093-fbd5afc69613 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:05.576056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:05.576928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:05.577210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:05.578045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] PATH_INFO: `/v3/users/f6b98dd545eb4ebb9c819f82103fae52` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:05.579236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6b98dd545eb4ebb9c819f82103fae52)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:05.581173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:05.608948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:05.626901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b98dd545eb4ebb9c819f82103fae52'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:05.627549 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b98dd545eb4ebb9c819f82103fae52'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:05.634544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b98dd545eb4ebb9c819f82103fae52'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:05.646942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b98dd545eb4ebb9c819f82103fae52'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:05.647568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b98dd545eb4ebb9c819f82103fae52'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:05.661000 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1235/2481] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:05 2022] DELETE /identity/v3/users/f6b98dd545eb4ebb9c819f82103fae52 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:05.665620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22189209-4077-473a-bc96-c134c841514c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:05.666107 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22189209-4077-473a-bc96-c134c841514c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:05.666227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22189209-4077-473a-bc96-c134c841514c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:05.683522 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-22189209-4077-473a-bc96-c134c841514c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:05.684717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-22189209-4077-473a-bc96-c134c841514c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:05.742860 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1247/2482] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:05.753032 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:05.754483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:05.812852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e11a41-090c-49b8-b46d-71fc243a2f84 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:05.816589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0093c-e5ec-4ada-b5ea-c054b68a93d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:05.817234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0093c-e5ec-4ada-b5ea-c054b68a93d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:05.817520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0093c-e5ec-4ada-b5ea-c054b68a93d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:05.817796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0093c-e5ec-4ada-b5ea-c054b68a93d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:05.826856 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0093c-e5ec-4ada-b5ea-c054b68a93d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:05.828848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0093c-e5ec-4ada-b5ea-c054b68a93d2 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:05.838269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1236/2483] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:06.108054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22189209-4077-473a-bc96-c134c841514c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:06.110243 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-22189209-4077-473a-bc96-c134c841514c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:06.131988 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22189209-4077-473a-bc96-c134c841514c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:06.137874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:06.139046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:06.139236 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:06.139669 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] PATH_INFO: `/v3/projects/5466cef7e1e842038f95082be46e9447` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:06.141396 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5466cef7e1e842038f95082be46e9447)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:06.143456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:06.199761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5466cef7e1e842038f95082be46e9447', 'request_id': 'req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:06.207386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5466cef7e1e842038f95082be46e9447', 'request_id': 'req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:06.214476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5466cef7e1e842038f95082be46e9447', 'request_id': 'req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:06.222819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:06.227183 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] The token cache is being invalidate because project 5466cef7e1e842038f95082be46e9447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:06.227783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:06.228742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c33c8ac5-7319-4ad0-91d3-4bef9f4c886b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:06.230859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1248/2484] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:06 2022] DELETE /identity/v3/projects/5466cef7e1e842038f95082be46e9447 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:06.236359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805b139-77ed-434e-b823-a8fd6ec7d5c9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:06.236744 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805b139-77ed-434e-b823-a8fd6ec7d5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:06.237073 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805b139-77ed-434e-b823-a8fd6ec7d5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:06.256556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1805b139-77ed-434e-b823-a8fd6ec7d5c9 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:06.257766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1805b139-77ed-434e-b823-a8fd6ec7d5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:06.324542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1237/2485] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:06.332235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81579aa1-3cac-4469-9f4c-909d9c0037ed None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:06.332529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81579aa1-3cac-4469-9f4c-909d9c0037ed None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:06.332828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81579aa1-3cac-4469-9f4c-909d9c0037ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:06.356881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-81579aa1-3cac-4469-9f4c-909d9c0037ed None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:06.358102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-81579aa1-3cac-4469-9f4c-909d9c0037ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:06.366774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1249/2486] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:06.371038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805b139-77ed-434e-b823-a8fd6ec7d5c9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:06.382360 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805b139-77ed-434e-b823-a8fd6ec7d5c9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:06.387277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c360a57-45cd-46e8-9516-c2b0fa993438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:06.388601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c360a57-45cd-46e8-9516-c2b0fa993438 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:06.388888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c360a57-45cd-46e8-9516-c2b0fa993438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:06.389168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c360a57-45cd-46e8-9516-c2b0fa993438 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:06.389929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c360a57-45cd-46e8-9516-c2b0fa993438 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:06.390321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c360a57-45cd-46e8-9516-c2b0fa993438 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:06.391936 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c360a57-45cd-46e8-9516-c2b0fa993438 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:06.407796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1238/2487] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:06.409013 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81579aa1-3cac-4469-9f4c-909d9c0037ed None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:06.418577 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81579aa1-3cac-4469-9f4c-909d9c0037ed None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:06.423980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a2b9-99ad-4f91-ae62-d0112b644337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:06.424636 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a2b9-99ad-4f91-ae62-d0112b644337 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:06.424922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a2b9-99ad-4f91-ae62-d0112b644337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:06.425230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a2b9-99ad-4f91-ae62-d0112b644337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:06.425928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9a2b9-99ad-4f91-ae62-d0112b644337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:06.427893 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9a2b9-99ad-4f91-ae62-d0112b644337 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:06.453874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1250/2488] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:06 2022] POST /identity/v3/projects => generated 403 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:06.458111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c360a57-45cd-46e8-9516-c2b0fa993438 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:06.468066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c360a57-45cd-46e8-9516-c2b0fa993438 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:06.501785 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:06.529255 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c7d14-3b90-4371-8d64-32822b20bb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:06.529916 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c7d14-3b90-4371-8d64-32822b20bb7d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:06.530233 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c7d14-3b90-4371-8d64-32822b20bb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:06.530507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c7d14-3b90-4371-8d64-32822b20bb7d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:06.531695 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c7d14-3b90-4371-8d64-32822b20bb7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:06.533656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c7d14-3b90-4371-8d64-32822b20bb7d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:06.553210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1239/2489] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:06 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:06.557599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a2b9-99ad-4f91-ae62-d0112b644337 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:06.568687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a2b9-99ad-4f91-ae62-d0112b644337 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:06.573319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7e817-8be4-48a5-8bf9-39e21ada49fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:06.574009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7e817-8be4-48a5-8bf9-39e21ada49fd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:06.574296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7e817-8be4-48a5-8bf9-39e21ada49fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:06.574576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7e817-8be4-48a5-8bf9-39e21ada49fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:06.575311 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7e817-8be4-48a5-8bf9-39e21ada49fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:06.576904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7e817-8be4-48a5-8bf9-39e21ada49fd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:06.588318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1251/2490] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:06.592212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c7d14-3b90-4371-8d64-32822b20bb7d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:06.602420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c7d14-3b90-4371-8d64-32822b20bb7d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:06.607027 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5561e-9e88-43d0-8f4b-cb3dd3c1459b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:06.607685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5561e-9e88-43d0-8f4b-cb3dd3c1459b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:06.607970 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5561e-9e88-43d0-8f4b-cb3dd3c1459b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:06.608250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5561e-9e88-43d0-8f4b-cb3dd3c1459b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:06.609055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5561e-9e88-43d0-8f4b-cb3dd3c1459b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:06.610774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5561e-9e88-43d0-8f4b-cb3dd3c1459b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:06.620500 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1240/2491] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:06.625001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7e817-8be4-48a5-8bf9-39e21ada49fd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:06.635325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7e817-8be4-48a5-8bf9-39e21ada49fd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:06.639846 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd979cf-b884-476c-84de-3e932c4c2848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:06.641873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd979cf-b884-476c-84de-3e932c4c2848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:06.642232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd979cf-b884-476c-84de-3e932c4c2848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:06.642507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd979cf-b884-476c-84de-3e932c4c2848 admin admin] PATH_INFO: `/v3/projects/fc88342bc3804d00a09e22006e087fbe/users/92dd857c54484de9afa603308246318b/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:06.657418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd979cf-b884-476c-84de-3e932c4c2848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc88342bc3804d00a09e22006e087fbe, user_id=92dd857c54484de9afa603308246318b, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:06.660112 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd979cf-b884-476c-84de-3e932c4c2848 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:06.673546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1252/2492] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:06 2022] PUT /identity/v3/projects/fc88342bc3804d00a09e22006e087fbe/users/92dd857c54484de9afa603308246318b/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:06.683554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693db26f-b5b4-4d6d-a736-cb71f931685e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:06.683852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693db26f-b5b4-4d6d-a736-cb71f931685e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:06.684138 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693db26f-b5b4-4d6d-a736-cb71f931685e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:06.702099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-693db26f-b5b4-4d6d-a736-cb71f931685e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:06.704092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-693db26f-b5b4-4d6d-a736-cb71f931685e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:06.762784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1241/2493] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:06.773081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd979cf-b884-476c-84de-3e932c4c2848 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:06.774914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5dd979cf-b884-476c-84de-3e932c4c2848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:06.832641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd979cf-b884-476c-84de-3e932c4c2848 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:06.836354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808dccf-d7c7-4813-82b0-7e5c3855cdb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:06.837009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808dccf-d7c7-4813-82b0-7e5c3855cdb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:06.837297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808dccf-d7c7-4813-82b0-7e5c3855cdb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:06.837598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808dccf-d7c7-4813-82b0-7e5c3855cdb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:06.847007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808dccf-d7c7-4813-82b0-7e5c3855cdb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:06.848433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808dccf-d7c7-4813-82b0-7e5c3855cdb4 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:06.857819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1253/2494] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:07.481456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc9815-8ed9-458c-b49e-d3ad9ffcaee0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:07.481755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc9815-8ed9-458c-b49e-d3ad9ffcaee0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:07.482110 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc9815-8ed9-458c-b49e-d3ad9ffcaee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:07.510081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c9fc9815-8ed9-458c-b49e-d3ad9ffcaee0 None None] MFA Rules not processed for user `92dd857c54484de9afa603308246318b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:07.511287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c9fc9815-8ed9-458c-b49e-d3ad9ffcaee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:07.602594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1242/2495] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:07.607084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808dccf-d7c7-4813-82b0-7e5c3855cdb4 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:07.617573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808dccf-d7c7-4813-82b0-7e5c3855cdb4 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:07.622198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ede53-7a93-4613-9eb4-50089521fea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:07.622855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ede53-7a93-4613-9eb4-50089521fea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:07.623156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ede53-7a93-4613-9eb4-50089521fea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:07.623431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ede53-7a93-4613-9eb4-50089521fea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:07.624179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ede53-7a93-4613-9eb4-50089521fea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:07.626359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ede53-7a93-4613-9eb4-50089521fea8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:07.654464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1254/2496] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:07 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:07.659266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc9815-8ed9-458c-b49e-d3ad9ffcaee0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:07.670221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc9815-8ed9-458c-b49e-d3ad9ffcaee0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:07.718229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:07.734945 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10540623-a3f7-4de7-bcc9-e219f31fc9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:07.735620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10540623-a3f7-4de7-bcc9-e219f31fc9f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:07.735907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10540623-a3f7-4de7-bcc9-e219f31fc9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:07.736188 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10540623-a3f7-4de7-bcc9-e219f31fc9f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:07.736881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10540623-a3f7-4de7-bcc9-e219f31fc9f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:07.738871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10540623-a3f7-4de7-bcc9-e219f31fc9f0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:07.760411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1243/2497] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:07 2022] POST /identity/v3/users => generated 309 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:07.764461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ede53-7a93-4613-9eb4-50089521fea8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:07.774906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ede53-7a93-4613-9eb4-50089521fea8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:07.779773 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dba47-b827-47fa-b044-4110051b4db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:07.780444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dba47-b827-47fa-b044-4110051b4db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:07.780780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dba47-b827-47fa-b044-4110051b4db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:07.781168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dba47-b827-47fa-b044-4110051b4db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:07.781980 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69dba47-b827-47fa-b044-4110051b4db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:07.783579 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69dba47-b827-47fa-b044-4110051b4db5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:07.794260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1255/2498] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:07.799207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10540623-a3f7-4de7-bcc9-e219f31fc9f0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:07.809600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10540623-a3f7-4de7-bcc9-e219f31fc9f0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:07.815368 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b55bc-40f8-4190-b37c-246d3cc9fa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:07.816205 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b55bc-40f8-4190-b37c-246d3cc9fa60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:07.816493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b55bc-40f8-4190-b37c-246d3cc9fa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:07.816775 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b55bc-40f8-4190-b37c-246d3cc9fa60 admin admin] PATH_INFO: `/v3/projects/9c843d5b03994766bf883552116d54c3/users/2c0a0fcbd431400780f3ea0b63288fe7/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:07.831676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b55bc-40f8-4190-b37c-246d3cc9fa60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c843d5b03994766bf883552116d54c3, user_id=2c0a0fcbd431400780f3ea0b63288fe7, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:07.834363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b55bc-40f8-4190-b37c-246d3cc9fa60 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:07.848204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1244/2499] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:07 2022] PUT /identity/v3/projects/9c843d5b03994766bf883552116d54c3/users/2c0a0fcbd431400780f3ea0b63288fe7/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:07.852647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dba47-b827-47fa-b044-4110051b4db5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:07.863434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dba47-b827-47fa-b044-4110051b4db5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:07.922816 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db2516-8bcb-4970-8159-e68fd70cc563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:07.923573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db2516-8bcb-4970-8159-e68fd70cc563 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:07.923811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db2516-8bcb-4970-8159-e68fd70cc563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:07.924254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db2516-8bcb-4970-8159-e68fd70cc563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:07.925055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db2516-8bcb-4970-8159-e68fd70cc563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:07.926678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db2516-8bcb-4970-8159-e68fd70cc563 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:07.936522 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1256/2500] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:07.941394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b55bc-40f8-4190-b37c-246d3cc9fa60 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:07.951790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b55bc-40f8-4190-b37c-246d3cc9fa60 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:07.956575 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a891d-2685-49d5-8139-5d01b36fdea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:07.957661 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a891d-2685-49d5-8139-5d01b36fdea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:07.957949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a891d-2685-49d5-8139-5d01b36fdea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:07.958267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a891d-2685-49d5-8139-5d01b36fdea8 admin admin] PATH_INFO: `/v3/projects/9c843d5b03994766bf883552116d54c3/users/2c0a0fcbd431400780f3ea0b63288fe7/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:07.960256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a891d-2685-49d5-8139-5d01b36fdea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c843d5b03994766bf883552116d54c3, user_id=2c0a0fcbd431400780f3ea0b63288fe7, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:07.962949 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a891d-2685-49d5-8139-5d01b36fdea8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:07.973733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1245/2501] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:07 2022] PUT /identity/v3/projects/9c843d5b03994766bf883552116d54c3/users/2c0a0fcbd431400780f3ea0b63288fe7/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:08.514010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb43491-6f99-4498-8c6a-99a6972a3bff None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:08.514305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb43491-6f99-4498-8c6a-99a6972a3bff None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:08.514587 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb43491-6f99-4498-8c6a-99a6972a3bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:08.520358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:08.535184 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cbb43491-6f99-4498-8c6a-99a6972a3bff None None] MFA Rules not processed for user `5439d1a65154486bb4fdfeeaf8ab4b07`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:08.536525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cbb43491-6f99-4498-8c6a-99a6972a3bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:08.600308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1257/2502] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:08.613530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a46deb-bfbe-41ef-af59-9bc96981e6dc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:08.614041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a46deb-bfbe-41ef-af59-9bc96981e6dc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:08.614347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a46deb-bfbe-41ef-af59-9bc96981e6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:08.649905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-17a46deb-bfbe-41ef-af59-9bc96981e6dc None None] MFA Rules not processed for user `2c0a0fcbd431400780f3ea0b63288fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:08.651687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-17a46deb-bfbe-41ef-af59-9bc96981e6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:08.770606 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1246/2503] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:08 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:08.822842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb43491-6f99-4498-8c6a-99a6972a3bff None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:08.826178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cbb43491-6f99-4498-8c6a-99a6972a3bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:08.924194 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb43491-6f99-4498-8c6a-99a6972a3bff None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:08.929102 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:08.930138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:08.930558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:08.930955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] PATH_INFO: `/v3/users/f1c6347473ab4ceeae5dcbcb707b5110` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:08.932260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c6347473ab4ceeae5dcbcb707b5110)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:08.934365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:08.996781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c6347473ab4ceeae5dcbcb707b5110'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:09.011587 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c6347473ab4ceeae5dcbcb707b5110'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:09.021900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c6347473ab4ceeae5dcbcb707b5110'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:09.022664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c6347473ab4ceeae5dcbcb707b5110'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:09.023280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c6347473ab4ceeae5dcbcb707b5110'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:09.042239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1258/2504] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:08 2022] DELETE /identity/v3/users/f1c6347473ab4ceeae5dcbcb707b5110 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:09.048345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33fa72-0f83-4a19-ae94-84a221cded05 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:09.048643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33fa72-0f83-4a19-ae94-84a221cded05 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:09.048928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33fa72-0f83-4a19-ae94-84a221cded05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:09.077018 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8f33fa72-0f83-4a19-ae94-84a221cded05 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:09.078874 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8f33fa72-0f83-4a19-ae94-84a221cded05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:09.146855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1247/2505] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:09.161043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:09.162943 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:09.200013 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:09.236065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d6753-6aa7-4c71-b0e1-cc541a8ea4dc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:09.240604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ca29b-61ec-48ad-a43e-874f9a3e7126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:09.241391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ca29b-61ec-48ad-a43e-874f9a3e7126 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:09.241753 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ca29b-61ec-48ad-a43e-874f9a3e7126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:09.242144 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ca29b-61ec-48ad-a43e-874f9a3e7126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:09.255348 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ca29b-61ec-48ad-a43e-874f9a3e7126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:09.257153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ca29b-61ec-48ad-a43e-874f9a3e7126 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:09.273553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1259/2506] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:09.680378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33fa72-0f83-4a19-ae94-84a221cded05 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:09.682670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8f33fa72-0f83-4a19-ae94-84a221cded05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:09.729625 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33fa72-0f83-4a19-ae94-84a221cded05 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:09.739338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:09.741572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:09.742117 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:09.742875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] PATH_INFO: `/v3/projects/4a95e572f9b14e2a84e6c4a4204277d0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:09.745288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a95e572f9b14e2a84e6c4a4204277d0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:09.748843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:09.786518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:09.838422 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a95e572f9b14e2a84e6c4a4204277d0', 'request_id': 'req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:09.845125 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a95e572f9b14e2a84e6c4a4204277d0', 'request_id': 'req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:09.853570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a95e572f9b14e2a84e6c4a4204277d0', 'request_id': 'req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:09.874150 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] The token cache is being invalidate because project 4a95e572f9b14e2a84e6c4a4204277d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:09.875075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:09.875843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:09.877448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1248/2507] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:09 2022] DELETE /identity/v3/projects/4a95e572f9b14e2a84e6c4a4204277d0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:09.881418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ca29b-61ec-48ad-a43e-874f9a3e7126 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:09.893141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ca29b-61ec-48ad-a43e-874f9a3e7126 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:09.965747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:09.966764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:09.967056 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:09.967335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] PATH_INFO: `/v3/users/f342354e01764850bdb169731851f432` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:09.969294 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f342354e01764850bdb169731851f432)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:09.971265 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:10.021915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f342354e01764850bdb169731851f432'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.038409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f342354e01764850bdb169731851f432'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.056314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f342354e01764850bdb169731851f432'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.057669 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f342354e01764850bdb169731851f432'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.058359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f342354e01764850bdb169731851f432'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.075063 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1260/2508] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:09 2022] DELETE /identity/v3/users/f342354e01764850bdb169731851f432 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:10.332573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:10.334083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:10.347881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5f03a-58c2-4126-b2c3-7a9326c6b7b1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:10.414185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:10.415106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:10.415412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:10.415664 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] PATH_INFO: `/v3/users/92dd857c54484de9afa603308246318b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:10.416870 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92dd857c54484de9afa603308246318b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:10.418819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:10.470995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92dd857c54484de9afa603308246318b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.471921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92dd857c54484de9afa603308246318b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.479567 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92dd857c54484de9afa603308246318b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.493298 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92dd857c54484de9afa603308246318b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.493619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92dd857c54484de9afa603308246318b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.509599 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1249/2509] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:10 2022] DELETE /identity/v3/users/92dd857c54484de9afa603308246318b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:10.557438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:10.559317 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:10.575045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded62d2a-2286-4a95-9f04-ee1a2a8ddb24 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:10.655152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:10.675095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:10.676091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:10.676442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:10.677237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] PATH_INFO: `/v3/projects/04b11ec86a254eecbff23c0efc50db25` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:10.678842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b11ec86a254eecbff23c0efc50db25)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:10.684037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:10.719345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:10.720970 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:10.740476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c43f1-b581-411f-a1e4-de6f59b7f1e3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:10.770562 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:10.774389 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b11ec86a254eecbff23c0efc50db25', 'request_id': 'req-bf2df5f5-211c-4935-b85e-a4540504816c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.789298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b11ec86a254eecbff23c0efc50db25', 'request_id': 'req-bf2df5f5-211c-4935-b85e-a4540504816c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.801801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b11ec86a254eecbff23c0efc50db25', 'request_id': 'req-bf2df5f5-211c-4935-b85e-a4540504816c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.822712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] The token cache is being invalidate because project 04b11ec86a254eecbff23c0efc50db25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:10.823253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.824157 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bf2df5f5-211c-4935-b85e-a4540504816c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.824285 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:10.825436 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:10.825802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:10.826111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] PATH_INFO: `/v3/projects/fc88342bc3804d00a09e22006e087fbe` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:10.826556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1261/2510] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:10 2022] DELETE /identity/v3/projects/04b11ec86a254eecbff23c0efc50db25 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:10.831207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc88342bc3804d00a09e22006e087fbe)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:10.832422 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5146ac-b6a8-4f73-a35c-10e196cd2af5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:10.832792 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5146ac-b6a8-4f73-a35c-10e196cd2af5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:10.833319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:10.833455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5146ac-b6a8-4f73-a35c-10e196cd2af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:10.857414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9e5146ac-b6a8-4f73-a35c-10e196cd2af5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:10.858669 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9e5146ac-b6a8-4f73-a35c-10e196cd2af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:10.875196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1262/2511] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:10.883513 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728834e3-76af-4546-8302-aa7382a45aa7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:10.894804 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc88342bc3804d00a09e22006e087fbe', 'request_id': 'req-74315476-7aac-4b5f-a487-342c7e4d17c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.898354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728834e3-76af-4546-8302-aa7382a45aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:10.898859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728834e3-76af-4546-8302-aa7382a45aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:10.900578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc88342bc3804d00a09e22006e087fbe', 'request_id': 'req-74315476-7aac-4b5f-a487-342c7e4d17c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.907948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc88342bc3804d00a09e22006e087fbe', 'request_id': 'req-74315476-7aac-4b5f-a487-342c7e4d17c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.951020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] The token cache is being invalidate because project fc88342bc3804d00a09e22006e087fbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:10.951290 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.952031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:10.955068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1250/2512] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:10 2022] DELETE /identity/v3/projects/fc88342bc3804d00a09e22006e087fbe => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:10.959126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:10.966411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:11.053872 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74315476-7aac-4b5f-a487-342c7e4d17c5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:11.058415 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-728834e3-76af-4546-8302-aa7382a45aa7 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:11.059368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-728834e3-76af-4546-8302-aa7382a45aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:11.060446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:11.061469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.061835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.062925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] PATH_INFO: `/v3/users/2c0a0fcbd431400780f3ea0b63288fe7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.064598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0a0fcbd431400780f3ea0b63288fe7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:11.066759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:11.073551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1263/2513] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:11.075829 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728834e3-76af-4546-8302-aa7382a45aa7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:11.088551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728834e3-76af-4546-8302-aa7382a45aa7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:11.095308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6be52-1e8a-4c61-9593-f38cfe67e636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:11.096155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6be52-1e8a-4c61-9593-f38cfe67e636 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.096550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6be52-1e8a-4c61-9593-f38cfe67e636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.096797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6be52-1e8a-4c61-9593-f38cfe67e636 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.097611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6be52-1e8a-4c61-9593-f38cfe67e636 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:11.097978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6be52-1e8a-4c61-9593-f38cfe67e636 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:11.099705 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6be52-1e8a-4c61-9593-f38cfe67e636 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:11.113443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1264/2514] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:11.118166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6be52-1e8a-4c61-9593-f38cfe67e636 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:11.123680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0a0fcbd431400780f3ea0b63288fe7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:11.124377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0a0fcbd431400780f3ea0b63288fe7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:11.129478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6be52-1e8a-4c61-9593-f38cfe67e636 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:11.133147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0a0fcbd431400780f3ea0b63288fe7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:11.134433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:11.135989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff16938-af92-4b96-a7a8-f862cfd1654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:11.137824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff16938-af92-4b96-a7a8-f862cfd1654a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.138217 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff16938-af92-4b96-a7a8-f862cfd1654a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.138611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff16938-af92-4b96-a7a8-f862cfd1654a admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.139281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff16938-af92-4b96-a7a8-f862cfd1654a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:11.142531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff16938-af92-4b96-a7a8-f862cfd1654a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:11.152467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0a0fcbd431400780f3ea0b63288fe7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:11.153284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0a0fcbd431400780f3ea0b63288fe7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:11.181640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1251/2515] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:10 2022] DELETE /identity/v3/users/2c0a0fcbd431400780f3ea0b63288fe7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:11.191237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1265/2516] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:11 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:38:11.192459 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:11.194050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:11.270357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e10c6e-cfd0-4f87-b7ae-035f5f925cdb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:11.348795 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2d06a-5544-4dee-9d68-09fdf5a1edea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:11.349713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2d06a-5544-4dee-9d68-09fdf5a1edea admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.350061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2d06a-5544-4dee-9d68-09fdf5a1edea admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.350370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2d06a-5544-4dee-9d68-09fdf5a1edea admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.351100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2d06a-5544-4dee-9d68-09fdf5a1edea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:11.353140 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2d06a-5544-4dee-9d68-09fdf5a1edea admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:11.371784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff16938-af92-4b96-a7a8-f862cfd1654a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:11.374109 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1ff16938-af92-4b96-a7a8-f862cfd1654a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:11.388513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1252/2517] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:11 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:38:11.389971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2d06a-5544-4dee-9d68-09fdf5a1edea admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:11.404231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff16938-af92-4b96-a7a8-f862cfd1654a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:11.405453 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2d06a-5544-4dee-9d68-09fdf5a1edea admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:11.411285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:11.413016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.413434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.413823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] PATH_INFO: `/v3/projects/9c843d5b03994766bf883552116d54c3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.414288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d539c24-2d1a-4efa-8b25-2f89784ab525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:11.415027 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d539c24-2d1a-4efa-8b25-2f89784ab525 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.415303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d539c24-2d1a-4efa-8b25-2f89784ab525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.415581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d539c24-2d1a-4efa-8b25-2f89784ab525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.415680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c843d5b03994766bf883552116d54c3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:11.416435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d539c24-2d1a-4efa-8b25-2f89784ab525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:11.417780 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:11.419375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d539c24-2d1a-4efa-8b25-2f89784ab525 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:11.434066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1253/2518] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:11.438493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d539c24-2d1a-4efa-8b25-2f89784ab525 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:11.450710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d539c24-2d1a-4efa-8b25-2f89784ab525 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:11.458754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f911d-64f9-4c29-8de4-3ecb60c8ff01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:11.459448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f911d-64f9-4c29-8de4-3ecb60c8ff01 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.459729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f911d-64f9-4c29-8de4-3ecb60c8ff01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.460009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f911d-64f9-4c29-8de4-3ecb60c8ff01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.460737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f911d-64f9-4c29-8de4-3ecb60c8ff01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:11.462546 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f911d-64f9-4c29-8de4-3ecb60c8ff01 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:11.483450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c843d5b03994766bf883552116d54c3', 'request_id': 'req-7123f912-00e8-4685-bf8f-b2a06b7496b1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:11.485055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1254/2519] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:11.486242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f911d-64f9-4c29-8de4-3ecb60c8ff01 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:11.494635 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c843d5b03994766bf883552116d54c3', 'request_id': 'req-7123f912-00e8-4685-bf8f-b2a06b7496b1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:11.504784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c843d5b03994766bf883552116d54c3', 'request_id': 'req-7123f912-00e8-4685-bf8f-b2a06b7496b1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:11.507509 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f911d-64f9-4c29-8de4-3ecb60c8ff01 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:11.523431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] The token cache is being invalidate because project 9c843d5b03994766bf883552116d54c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:11.523725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:11.524520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7123f912-00e8-4685-bf8f-b2a06b7496b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:11.528723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1266/2520] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:11 2022] DELETE /identity/v3/projects/9c843d5b03994766bf883552116d54c3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:11.535123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2f875-3154-4724-b262-9d8deff73153 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.535878 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2f875-3154-4724-b262-9d8deff73153 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.536023 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2f875-3154-4724-b262-9d8deff73153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.569826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f2b2f875-3154-4724-b262-9d8deff73153 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:11.572884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f2b2f875-3154-4724-b262-9d8deff73153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:11.582331 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:11.586777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:11.615050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e072f-359e-4fb4-a418-d5fa66ae0e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:11.616943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e072f-359e-4fb4-a418-d5fa66ae0e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.617352 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e072f-359e-4fb4-a418-d5fa66ae0e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.617757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e072f-359e-4fb4-a418-d5fa66ae0e2b admin admin] PATH_INFO: `/v3/projects/5a4681a07f154a268c06f7c26cf66db1/users/a4990a985c4e45c4b1b07960d98f0923/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.674554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e072f-359e-4fb4-a418-d5fa66ae0e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4681a07f154a268c06f7c26cf66db1, user_id=a4990a985c4e45c4b1b07960d98f0923, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:11.677791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e072f-359e-4fb4-a418-d5fa66ae0e2b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:11.682234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1267/2521] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:11.693819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5c3da-5366-4c42-bb53-551793ceb37e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.694360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5c3da-5366-4c42-bb53-551793ceb37e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.694798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5c3da-5366-4c42-bb53-551793ceb37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.703689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1255/2522] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:11 2022] PUT /identity/v3/projects/5a4681a07f154a268c06f7c26cf66db1/users/a4990a985c4e45c4b1b07960d98f0923/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:11.717073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-14d5c3da-5366-4c42-bb53-551793ceb37e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:11.717547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3920ef-6c0f-4e55-be25-e601bd20f891 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.717932 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3920ef-6c0f-4e55-be25-e601bd20f891 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.718869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3920ef-6c0f-4e55-be25-e601bd20f891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.720223 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-14d5c3da-5366-4c42-bb53-551793ceb37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:11.756531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6f3920ef-6c0f-4e55-be25-e601bd20f891 None None] MFA Rules not processed for user `a4990a985c4e45c4b1b07960d98f0923`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:11.757433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6f3920ef-6c0f-4e55-be25-e601bd20f891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:11.813979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1268/2523] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:11.818931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5c3da-5366-4c42-bb53-551793ceb37e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:11.834351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5c3da-5366-4c42-bb53-551793ceb37e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:11.840064 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b6f5a-89cb-4dca-9f4e-eac34e70080f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:11.840876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b6f5a-89cb-4dca-9f4e-eac34e70080f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.841266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b6f5a-89cb-4dca-9f4e-eac34e70080f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.841651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b6f5a-89cb-4dca-9f4e-eac34e70080f admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.842538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b6f5a-89cb-4dca-9f4e-eac34e70080f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:11.842956 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b6f5a-89cb-4dca-9f4e-eac34e70080f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:11.845390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b6f5a-89cb-4dca-9f4e-eac34e70080f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:11.854138 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1256/2524] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:11.863965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3920ef-6c0f-4e55-be25-e601bd20f891 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:11.865996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6f3920ef-6c0f-4e55-be25-e601bd20f891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:11.868277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1269/2525] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:11.872469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b6f5a-89cb-4dca-9f4e-eac34e70080f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:11.883880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b6f5a-89cb-4dca-9f4e-eac34e70080f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:11.888785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582615b-b6b8-48c6-83b0-c27fb9e06cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:11.889521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582615b-b6b8-48c6-83b0-c27fb9e06cdf admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:11.889875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582615b-b6b8-48c6-83b0-c27fb9e06cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:11.891237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582615b-b6b8-48c6-83b0-c27fb9e06cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:11.891915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582615b-b6b8-48c6-83b0-c27fb9e06cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:11.894071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582615b-b6b8-48c6-83b0-c27fb9e06cdf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:11.925752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1270/2526] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:11 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:38:11.929468 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582615b-b6b8-48c6-83b0-c27fb9e06cdf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:11.932652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3920ef-6c0f-4e55-be25-e601bd20f891 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:11.996358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582615b-b6b8-48c6-83b0-c27fb9e06cdf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:12.001332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7f236-10a1-44b2-b2e1-65fb79f4833f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:12.002280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7f236-10a1-44b2-b2e1-65fb79f4833f service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.002666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7f236-10a1-44b2-b2e1-65fb79f4833f service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.002844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa6d0b-a686-413a-92fc-3c1ed7697a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:12.003149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7f236-10a1-44b2-b2e1-65fb79f4833f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.003593 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa6d0b-a686-413a-92fc-3c1ed7697a56 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.003886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa6d0b-a686-413a-92fc-3c1ed7697a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.004168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa6d0b-a686-413a-92fc-3c1ed7697a56 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.004883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa6d0b-a686-413a-92fc-3c1ed7697a56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:12.006990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa6d0b-a686-413a-92fc-3c1ed7697a56 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:12.030902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1271/2527] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:11 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:38:12.034903 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa6d0b-a686-413a-92fc-3c1ed7697a56 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:12.045683 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa6d0b-a686-413a-92fc-3c1ed7697a56 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:12.049019 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:12.052106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a47f7-029d-484a-bc5d-51b7136d93c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:12.059312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a47f7-029d-484a-bc5d-51b7136d93c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.059610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a47f7-029d-484a-bc5d-51b7136d93c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.059898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a47f7-029d-484a-bc5d-51b7136d93c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.060659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a47f7-029d-484a-bc5d-51b7136d93c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:12.063035 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a47f7-029d-484a-bc5d-51b7136d93c7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:12.065715 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:12.077014 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1272/2528] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:38:12.081078 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a47f7-029d-484a-bc5d-51b7136d93c7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:12.091439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a47f7-029d-484a-bc5d-51b7136d93c7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:12.092066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7f236-10a1-44b2-b2e1-65fb79f4833f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:12.094511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7f236-10a1-44b2-b2e1-65fb79f4833f service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:12.096428 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d49902-b924-4ce3-8529-e00a02b81d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:12.097253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d49902-b924-4ce3-8529-e00a02b81d51 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.097537 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d49902-b924-4ce3-8529-e00a02b81d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.097820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d49902-b924-4ce3-8529-e00a02b81d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.098785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d49902-b924-4ce3-8529-e00a02b81d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:12.100376 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d49902-b924-4ce3-8529-e00a02b81d51 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:12.107891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1257/2529] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:38:12.120772 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1273/2530] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:38:12.122631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7f236-10a1-44b2-b2e1-65fb79f4833f service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:12.138147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7f236-10a1-44b2-b2e1-65fb79f4833f service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:12.144163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb867f7d-730e-4b27-86ef-c2d7d6e78f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:12.145405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb867f7d-730e-4b27-86ef-c2d7d6e78f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.145815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb867f7d-730e-4b27-86ef-c2d7d6e78f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.146122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb867f7d-730e-4b27-86ef-c2d7d6e78f94 admin admin] PATH_INFO: `/v3/projects/cc4e4caddfd745e4953005676e9999d6/users/b1aa7a6d3eec40689bb060084d4f6877/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.165500 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb867f7d-730e-4b27-86ef-c2d7d6e78f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc4e4caddfd745e4953005676e9999d6, user_id=b1aa7a6d3eec40689bb060084d4f6877, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:12.168782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb867f7d-730e-4b27-86ef-c2d7d6e78f94 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:12.183965 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1258/2531] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:12 2022] PUT /identity/v3/projects/cc4e4caddfd745e4953005676e9999d6/users/b1aa7a6d3eec40689bb060084d4f6877/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:12.195826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd75ba-3a46-48fa-a54b-d40615eb4584 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.196114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd75ba-3a46-48fa-a54b-d40615eb4584 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.196395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd75ba-3a46-48fa-a54b-d40615eb4584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.230663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c9cd75ba-3a46-48fa-a54b-d40615eb4584 None None] MFA Rules not processed for user `b1aa7a6d3eec40689bb060084d4f6877`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:12.231952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c9cd75ba-3a46-48fa-a54b-d40615eb4584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:12.313976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1274/2532] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:12.318306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb867f7d-730e-4b27-86ef-c2d7d6e78f94 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:12.431220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb867f7d-730e-4b27-86ef-c2d7d6e78f94 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:12.438266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783d55d-c041-4d28-b184-2803d9d3e403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:12.445991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783d55d-c041-4d28-b184-2803d9d3e403 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.445991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783d55d-c041-4d28-b184-2803d9d3e403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.445991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783d55d-c041-4d28-b184-2803d9d3e403 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.446727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783d55d-c041-4d28-b184-2803d9d3e403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:12.453468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783d55d-c041-4d28-b184-2803d9d3e403 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:12.519450 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1259/2533] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:12 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:12.523007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd75ba-3a46-48fa-a54b-d40615eb4584 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:12.537890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd75ba-3a46-48fa-a54b-d40615eb4584 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:12.625686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4a110-80da-4afa-ae28-be6c757d111a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:12.626473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4a110-80da-4afa-ae28-be6c757d111a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.626753 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4a110-80da-4afa-ae28-be6c757d111a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.627092 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4a110-80da-4afa-ae28-be6c757d111a admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.627727 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4a110-80da-4afa-ae28-be6c757d111a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:12.629766 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4a110-80da-4afa-ae28-be6c757d111a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:12.658326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1275/2534] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:12 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:12.662115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783d55d-c041-4d28-b184-2803d9d3e403 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:12.689728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783d55d-c041-4d28-b184-2803d9d3e403 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:12.712801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd974294-61b1-4659-b65a-4c56b7e0661f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:12.715600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd974294-61b1-4659-b65a-4c56b7e0661f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.715907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd974294-61b1-4659-b65a-4c56b7e0661f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.716192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd974294-61b1-4659-b65a-4c56b7e0661f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.718245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd974294-61b1-4659-b65a-4c56b7e0661f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:12.721004 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd974294-61b1-4659-b65a-4c56b7e0661f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:12.739238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4a110-80da-4afa-ae28-be6c757d111a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:12.742088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1260/2535] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:38:12.756701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4a110-80da-4afa-ae28-be6c757d111a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:12.762065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2ae73-b5b0-49e6-99e5-0c59cfa51733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:12.762982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2ae73-b5b0-49e6-99e5-0c59cfa51733 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.763267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2ae73-b5b0-49e6-99e5-0c59cfa51733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.763901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2ae73-b5b0-49e6-99e5-0c59cfa51733 admin admin] PATH_INFO: `/v3/projects/df248c78e48049db8b728f17639dedea/users/041fcd377980477fba51b54a4c6f3038/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.783978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2ae73-b5b0-49e6-99e5-0c59cfa51733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df248c78e48049db8b728f17639dedea, user_id=041fcd377980477fba51b54a4c6f3038, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:12.786834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2ae73-b5b0-49e6-99e5-0c59cfa51733 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:12.806483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1276/2536] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:12 2022] PUT /identity/v3/projects/df248c78e48049db8b728f17639dedea/users/041fcd377980477fba51b54a4c6f3038/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:12.808897 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd974294-61b1-4659-b65a-4c56b7e0661f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:12.822985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd974294-61b1-4659-b65a-4c56b7e0661f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:12.893588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8a300-95e2-410a-972a-92c599be2e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:12.894354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8a300-95e2-410a-972a-92c599be2e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.894638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8a300-95e2-410a-972a-92c599be2e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.894942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8a300-95e2-410a-972a-92c599be2e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.895678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8a300-95e2-410a-972a-92c599be2e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:12.897343 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8a300-95e2-410a-972a-92c599be2e42 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:12.908950 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1261/2537] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:12.913414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2ae73-b5b0-49e6-99e5-0c59cfa51733 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:12.974628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:12.977705 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2ae73-b5b0-49e6-99e5-0c59cfa51733 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:12.984061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f070a7-7017-43a6-a81d-4089e5729349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:12.985005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f070a7-7017-43a6-a81d-4089e5729349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:12.985264 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f070a7-7017-43a6-a81d-4089e5729349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:12.985533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f070a7-7017-43a6-a81d-4089e5729349 admin admin] PATH_INFO: `/v3/projects/df248c78e48049db8b728f17639dedea/users/041fcd377980477fba51b54a4c6f3038/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:12.987838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f070a7-7017-43a6-a81d-4089e5729349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df248c78e48049db8b728f17639dedea, user_id=041fcd377980477fba51b54a4c6f3038, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:12.990732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f070a7-7017-43a6-a81d-4089e5729349 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:13.026016 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1277/2538] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:12 2022] PUT /identity/v3/projects/df248c78e48049db8b728f17639dedea/users/041fcd377980477fba51b54a4c6f3038/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:13.028274 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eff14-de85-4162-9b0e-68bc3615e97d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:13.028564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eff14-de85-4162-9b0e-68bc3615e97d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:13.028842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eff14-de85-4162-9b0e-68bc3615e97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:13.059002 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:13.070560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-828eff14-de85-4162-9b0e-68bc3615e97d None None] MFA Rules not processed for user `041fcd377980477fba51b54a4c6f3038`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:13.071760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-828eff14-de85-4162-9b0e-68bc3615e97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:13.139420 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1262/2539] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:13.151541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f070a7-7017-43a6-a81d-4089e5729349 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:13.221305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f070a7-7017-43a6-a81d-4089e5729349 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:13.224874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231478a-a0ee-4d4c-bd01-7af3e3e43926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:13.225522 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231478a-a0ee-4d4c-bd01-7af3e3e43926 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:13.225802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231478a-a0ee-4d4c-bd01-7af3e3e43926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:13.226124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231478a-a0ee-4d4c-bd01-7af3e3e43926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:13.235275 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231478a-a0ee-4d4c-bd01-7af3e3e43926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:13.236692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231478a-a0ee-4d4c-bd01-7af3e3e43926 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:13.246064 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1278/2540] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:13.656432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eff14-de85-4162-9b0e-68bc3615e97d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:13.738961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eff14-de85-4162-9b0e-68bc3615e97d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:13.745067 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:13.746013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:13.746302 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:13.746579 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] PATH_INFO: `/v3/users/b1aa7a6d3eec40689bb060084d4f6877` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:13.747984 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1aa7a6d3eec40689bb060084d4f6877)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:13.750840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:13.812872 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1aa7a6d3eec40689bb060084d4f6877'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:13.813592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1aa7a6d3eec40689bb060084d4f6877'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:13.821818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1aa7a6d3eec40689bb060084d4f6877'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:13.835591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1aa7a6d3eec40689bb060084d4f6877'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:13.836338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1aa7a6d3eec40689bb060084d4f6877'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:13.854376 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1263/2541] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:13 2022] DELETE /identity/v3/users/b1aa7a6d3eec40689bb060084d4f6877 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:13.859851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f5ae9-2df3-4400-b833-8ecb01109e7c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:13.860170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f5ae9-2df3-4400-b833-8ecb01109e7c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:13.860453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f5ae9-2df3-4400-b833-8ecb01109e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:13.887755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c59f5ae9-2df3-4400-b833-8ecb01109e7c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:13.889007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c59f5ae9-2df3-4400-b833-8ecb01109e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:13.953383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1279/2542] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:13.963156 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:13.964589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:13.980191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:14.029382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b85ba5-8ced-40ea-bac7-29d1b86d56eb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:14.033210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05a8c6-0ee6-447b-8897-cb19642e4f7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:14.034552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05a8c6-0ee6-447b-8897-cb19642e4f7b service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:14.034843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05a8c6-0ee6-447b-8897-cb19642e4f7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:14.035121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05a8c6-0ee6-447b-8897-cb19642e4f7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:14.045505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05a8c6-0ee6-447b-8897-cb19642e4f7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:14.047022 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05a8c6-0ee6-447b-8897-cb19642e4f7b service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:14.060047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1264/2543] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:14.433817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f5ae9-2df3-4400-b833-8ecb01109e7c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:14.435259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c59f5ae9-2df3-4400-b833-8ecb01109e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:14.450429 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f5ae9-2df3-4400-b833-8ecb01109e7c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:14.455150 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:14.456044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:14.456327 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:14.456602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] PATH_INFO: `/v3/projects/cc4e4caddfd745e4953005676e9999d6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:14.457793 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc4e4caddfd745e4953005676e9999d6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:14.459848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:14.512064 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc4e4caddfd745e4953005676e9999d6', 'request_id': 'req-2f77d0ff-270e-4ac5-8803-f704ab1811b0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:14.517173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:14.521765 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc4e4caddfd745e4953005676e9999d6', 'request_id': 'req-2f77d0ff-270e-4ac5-8803-f704ab1811b0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:14.531978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc4e4caddfd745e4953005676e9999d6', 'request_id': 'req-2f77d0ff-270e-4ac5-8803-f704ab1811b0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:14.543065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] The token cache is being invalidate because project cc4e4caddfd745e4953005676e9999d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:14.543357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:14.544043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:14.545647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1280/2544] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:14 2022] DELETE /identity/v3/projects/cc4e4caddfd745e4953005676e9999d6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:14.550123 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05a8c6-0ee6-447b-8897-cb19642e4f7b service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:14.560057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05a8c6-0ee6-447b-8897-cb19642e4f7b service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:14.621802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ebecd-2dda-486f-bf48-6ee6037c2796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:14.622623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ebecd-2dda-486f-bf48-6ee6037c2796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:14.622908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ebecd-2dda-486f-bf48-6ee6037c2796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:14.623187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ebecd-2dda-486f-bf48-6ee6037c2796 admin admin] PATH_INFO: `/v3/users/041fcd377980477fba51b54a4c6f3038` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:14.624384 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ebecd-2dda-486f-bf48-6ee6037c2796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=041fcd377980477fba51b54a4c6f3038)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:14.627154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ebecd-2dda-486f-bf48-6ee6037c2796 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:14.675770 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-317ebecd-2dda-486f-bf48-6ee6037c2796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041fcd377980477fba51b54a4c6f3038'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:14.676414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-317ebecd-2dda-486f-bf48-6ee6037c2796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041fcd377980477fba51b54a4c6f3038'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:14.683483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-317ebecd-2dda-486f-bf48-6ee6037c2796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041fcd377980477fba51b54a4c6f3038'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:14.695614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-317ebecd-2dda-486f-bf48-6ee6037c2796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041fcd377980477fba51b54a4c6f3038'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:14.696273 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-317ebecd-2dda-486f-bf48-6ee6037c2796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041fcd377980477fba51b54a4c6f3038'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:14.709629 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1265/2545] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:14 2022] DELETE /identity/v3/users/041fcd377980477fba51b54a4c6f3038 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:15.017522 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:15.018984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:15.032237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77d0ff-270e-4ac5-8803-f704ab1811b0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:15.090276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:15.091158 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.091440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.091714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] PATH_INFO: `/v3/projects/df248c78e48049db8b728f17639dedea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.092895 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df248c78e48049db8b728f17639dedea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:15.095066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:15.140011 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df248c78e48049db8b728f17639dedea', 'request_id': 'req-e924a6d5-b5d4-4f45-bef3-28261eb43e33'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:15.146901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df248c78e48049db8b728f17639dedea', 'request_id': 'req-e924a6d5-b5d4-4f45-bef3-28261eb43e33'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:15.153160 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df248c78e48049db8b728f17639dedea', 'request_id': 'req-e924a6d5-b5d4-4f45-bef3-28261eb43e33'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:15.163508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] The token cache is being invalidate because project df248c78e48049db8b728f17639dedea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:15.163736 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:15.164888 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e924a6d5-b5d4-4f45-bef3-28261eb43e33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:15.166427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1281/2546] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:15 2022] DELETE /identity/v3/projects/df248c78e48049db8b728f17639dedea => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:15.171935 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ec52c-544c-4188-9732-708709bd7da5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.172395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ec52c-544c-4188-9732-708709bd7da5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.172395 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ec52c-544c-4188-9732-708709bd7da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.219325 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4b6ec52c-544c-4188-9732-708709bd7da5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:15.220561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4b6ec52c-544c-4188-9732-708709bd7da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:15.225914 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:15.290841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1266/2547] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:15.300508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df05a5c-d0f1-4546-bc58-2594634c7568 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.300800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df05a5c-d0f1-4546-bc58-2594634c7568 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.301084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df05a5c-d0f1-4546-bc58-2594634c7568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.319874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0df05a5c-d0f1-4546-bc58-2594634c7568 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:15.321083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0df05a5c-d0f1-4546-bc58-2594634c7568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:15.329330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1282/2548] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:15.333546 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ec52c-544c-4188-9732-708709bd7da5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:15.343517 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ec52c-544c-4188-9732-708709bd7da5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:15.348380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37970fd3-07f2-42b7-81a6-cfa1b075b330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:15.349044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37970fd3-07f2-42b7-81a6-cfa1b075b330 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.349333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37970fd3-07f2-42b7-81a6-cfa1b075b330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.349615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37970fd3-07f2-42b7-81a6-cfa1b075b330 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.350391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37970fd3-07f2-42b7-81a6-cfa1b075b330 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:15.350702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37970fd3-07f2-42b7-81a6-cfa1b075b330 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:15.352304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37970fd3-07f2-42b7-81a6-cfa1b075b330 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:15.365602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1267/2549] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:15.370129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df05a5c-d0f1-4546-bc58-2594634c7568 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:15.380285 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df05a5c-d0f1-4546-bc58-2594634c7568 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:15.385220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8e2be-3d11-47b6-aecc-1713e0acc463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:15.386095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8e2be-3d11-47b6-aecc-1713e0acc463 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.386379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8e2be-3d11-47b6-aecc-1713e0acc463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.386654 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8e2be-3d11-47b6-aecc-1713e0acc463 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.387464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8e2be-3d11-47b6-aecc-1713e0acc463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:15.389367 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8e2be-3d11-47b6-aecc-1713e0acc463 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:15.415701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1283/2550] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:15 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:15.419855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37970fd3-07f2-42b7-81a6-cfa1b075b330 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:15.431295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37970fd3-07f2-42b7-81a6-cfa1b075b330 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:15.492569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e032a07-2a4a-4fd2-a8a6-2236fa2ce49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:15.493233 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e032a07-2a4a-4fd2-a8a6-2236fa2ce49a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.493523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e032a07-2a4a-4fd2-a8a6-2236fa2ce49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.493804 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e032a07-2a4a-4fd2-a8a6-2236fa2ce49a admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.494529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e032a07-2a4a-4fd2-a8a6-2236fa2ce49a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:15.496451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e032a07-2a4a-4fd2-a8a6-2236fa2ce49a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:15.516421 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1268/2551] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:15 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:15.520569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8e2be-3d11-47b6-aecc-1713e0acc463 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:15.531512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8e2be-3d11-47b6-aecc-1713e0acc463 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:15.536089 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723582e4-b75f-40cf-a957-87bf25cebf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:15.536743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723582e4-b75f-40cf-a957-87bf25cebf78 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.537023 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723582e4-b75f-40cf-a957-87bf25cebf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.537298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723582e4-b75f-40cf-a957-87bf25cebf78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.538061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723582e4-b75f-40cf-a957-87bf25cebf78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:15.539640 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723582e4-b75f-40cf-a957-87bf25cebf78 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:15.551301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1284/2552] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:15.555143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e032a07-2a4a-4fd2-a8a6-2236fa2ce49a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:15.565232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e032a07-2a4a-4fd2-a8a6-2236fa2ce49a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:15.569808 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccfeaf-a490-4b5c-a16f-1a07fbdf5141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:15.570510 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccfeaf-a490-4b5c-a16f-1a07fbdf5141 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.570872 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccfeaf-a490-4b5c-a16f-1a07fbdf5141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.571305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccfeaf-a490-4b5c-a16f-1a07fbdf5141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.572132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccfeaf-a490-4b5c-a16f-1a07fbdf5141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:15.573711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccfeaf-a490-4b5c-a16f-1a07fbdf5141 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:15.583681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1269/2553] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:15.587675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723582e4-b75f-40cf-a957-87bf25cebf78 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:15.598113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723582e4-b75f-40cf-a957-87bf25cebf78 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:15.602689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5751a6-41a3-4b03-aed8-b7c7464f6b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:15.603561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5751a6-41a3-4b03-aed8-b7c7464f6b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.603840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5751a6-41a3-4b03-aed8-b7c7464f6b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.604113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5751a6-41a3-4b03-aed8-b7c7464f6b05 admin admin] PATH_INFO: `/v3/projects/861a750092db47f8868e4bef039adc28/users/18d6f0253aa44f069db625dde517fb18/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.619215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5751a6-41a3-4b03-aed8-b7c7464f6b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=861a750092db47f8868e4bef039adc28, user_id=18d6f0253aa44f069db625dde517fb18, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:15.621809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5751a6-41a3-4b03-aed8-b7c7464f6b05 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:15.634594 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1285/2554] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:15 2022] PUT /identity/v3/projects/861a750092db47f8868e4bef039adc28/users/18d6f0253aa44f069db625dde517fb18/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:15.644743 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc5c0b-bbd1-4572-ba37-03c673a551fb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.645040 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc5c0b-bbd1-4572-ba37-03c673a551fb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.645323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc5c0b-bbd1-4572-ba37-03c673a551fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.673283 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-94cc5c0b-bbd1-4572-ba37-03c673a551fb None None] MFA Rules not processed for user `18d6f0253aa44f069db625dde517fb18`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:15.674442 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-94cc5c0b-bbd1-4572-ba37-03c673a551fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:15.741876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1270/2555] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:15.746358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5751a6-41a3-4b03-aed8-b7c7464f6b05 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:15.757503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:15.818401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5751a6-41a3-4b03-aed8-b7c7464f6b05 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:15.823733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb2e84-327d-4b47-8f4c-f2fc7848f93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:15.824401 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb2e84-327d-4b47-8f4c-f2fc7848f93b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.824685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb2e84-327d-4b47-8f4c-f2fc7848f93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.824962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb2e84-327d-4b47-8f4c-f2fc7848f93b admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.826031 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb2e84-327d-4b47-8f4c-f2fc7848f93b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:15.828416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb2e84-327d-4b47-8f4c-f2fc7848f93b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:15.855180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1286/2556] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:15 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:15.859809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc5c0b-bbd1-4572-ba37-03c673a551fb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:15.870886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc5c0b-bbd1-4572-ba37-03c673a551fb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:15.875747 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:15.931262 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dfe50-c4e2-4a94-a151-b4bd9935837e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:15.931859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dfe50-c4e2-4a94-a151-b4bd9935837e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.932143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dfe50-c4e2-4a94-a151-b4bd9935837e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.932425 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dfe50-c4e2-4a94-a151-b4bd9935837e admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.933109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dfe50-c4e2-4a94-a151-b4bd9935837e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:15.935059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dfe50-c4e2-4a94-a151-b4bd9935837e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:15.955728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1271/2557] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:15 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:15.959745 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb2e84-327d-4b47-8f4c-f2fc7848f93b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:15.970862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb2e84-327d-4b47-8f4c-f2fc7848f93b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:15.975434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f01f6-fc67-464e-ab78-f4f09ee63980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:15.976086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f01f6-fc67-464e-ab78-f4f09ee63980 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:15.976368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f01f6-fc67-464e-ab78-f4f09ee63980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:15.976641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f01f6-fc67-464e-ab78-f4f09ee63980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:15.977375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f01f6-fc67-464e-ab78-f4f09ee63980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:15.978984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f01f6-fc67-464e-ab78-f4f09ee63980 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:15.989897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1287/2558] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:15.994044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dfe50-c4e2-4a94-a151-b4bd9935837e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:16.004561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dfe50-c4e2-4a94-a151-b4bd9935837e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:16.009094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2d1d8-b702-461c-b45c-4484094efeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:16.009987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2d1d8-b702-461c-b45c-4484094efeaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:16.010272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2d1d8-b702-461c-b45c-4484094efeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:16.010547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2d1d8-b702-461c-b45c-4484094efeaf admin admin] PATH_INFO: `/v3/projects/8361558c32f64b7fa2b9b7644fa4c3b5/users/f74a22c40d1c40aa808c84381c2fa355/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:16.024891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2d1d8-b702-461c-b45c-4484094efeaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8361558c32f64b7fa2b9b7644fa4c3b5, user_id=f74a22c40d1c40aa808c84381c2fa355, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:16.027816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2d1d8-b702-461c-b45c-4484094efeaf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:16.040822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1272/2559] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:15 2022] PUT /identity/v3/projects/8361558c32f64b7fa2b9b7644fa4c3b5/users/f74a22c40d1c40aa808c84381c2fa355/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:16.044664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f01f6-fc67-464e-ab78-f4f09ee63980 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:16.055838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f01f6-fc67-464e-ab78-f4f09ee63980 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:16.112882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c1c0a3-ebfe-43e5-b728-7bac9cd553c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:16.113548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1c0a3-ebfe-43e5-b728-7bac9cd553c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:16.113838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1c0a3-ebfe-43e5-b728-7bac9cd553c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:16.114149 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1c0a3-ebfe-43e5-b728-7bac9cd553c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:16.114878 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c1c0a3-ebfe-43e5-b728-7bac9cd553c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:16.116460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c1c0a3-ebfe-43e5-b728-7bac9cd553c7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:16.126946 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1288/2560] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:16.132825 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2d1d8-b702-461c-b45c-4484094efeaf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:16.143196 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2d1d8-b702-461c-b45c-4484094efeaf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:16.147817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba882c45-51f9-4813-a9ab-4f3b0af3cd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:16.148689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba882c45-51f9-4813-a9ab-4f3b0af3cd37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:16.148974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba882c45-51f9-4813-a9ab-4f3b0af3cd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:16.149253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba882c45-51f9-4813-a9ab-4f3b0af3cd37 admin admin] PATH_INFO: `/v3/projects/8361558c32f64b7fa2b9b7644fa4c3b5/users/f74a22c40d1c40aa808c84381c2fa355/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:16.151234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba882c45-51f9-4813-a9ab-4f3b0af3cd37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8361558c32f64b7fa2b9b7644fa4c3b5, user_id=f74a22c40d1c40aa808c84381c2fa355, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:16.154147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba882c45-51f9-4813-a9ab-4f3b0af3cd37 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:16.164680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1273/2561] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:16 2022] PUT /identity/v3/projects/8361558c32f64b7fa2b9b7644fa4c3b5/users/f74a22c40d1c40aa808c84381c2fa355/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:16.175257 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679809a-0dac-4bf7-9549-77ad3fb918dc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:16.175555 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679809a-0dac-4bf7-9549-77ad3fb918dc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:16.175836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679809a-0dac-4bf7-9549-77ad3fb918dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:16.202414 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9679809a-0dac-4bf7-9549-77ad3fb918dc None None] MFA Rules not processed for user `f74a22c40d1c40aa808c84381c2fa355`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:16.203705 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9679809a-0dac-4bf7-9549-77ad3fb918dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:16.269949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1289/2562] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:16.281716 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba882c45-51f9-4813-a9ab-4f3b0af3cd37 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:16.283454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ba882c45-51f9-4813-a9ab-4f3b0af3cd37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:16.339158 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba882c45-51f9-4813-a9ab-4f3b0af3cd37 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:16.342732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f124e39-322c-4aa5-abdd-8ebf94072afa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:16.343868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f124e39-322c-4aa5-abdd-8ebf94072afa service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:16.344151 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f124e39-322c-4aa5-abdd-8ebf94072afa service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:16.344425 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f124e39-322c-4aa5-abdd-8ebf94072afa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:16.353213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f124e39-322c-4aa5-abdd-8ebf94072afa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:16.354656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f124e39-322c-4aa5-abdd-8ebf94072afa service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:16.364494 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1274/2563] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:16.434309 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679809a-0dac-4bf7-9549-77ad3fb918dc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:16.440645 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:16.444735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679809a-0dac-4bf7-9549-77ad3fb918dc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:16.449681 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5255e7-a171-4375-b9d1-029d6753b4e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:16.450361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5255e7-a171-4375-b9d1-029d6753b4e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:16.450644 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5255e7-a171-4375-b9d1-029d6753b4e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:16.450922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5255e7-a171-4375-b9d1-029d6753b4e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:16.510181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5255e7-a171-4375-b9d1-029d6753b4e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:16.511619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5255e7-a171-4375-b9d1-029d6753b4e0 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:16.520828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1290/2564] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:16 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:19.420109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f124e39-322c-4aa5-abdd-8ebf94072afa service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:19.488490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f124e39-322c-4aa5-abdd-8ebf94072afa service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:19.494115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:19.494407 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:19.494687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:19.494968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] PATH_INFO: `/v3/users/18d6f0253aa44f069db625dde517fb18` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:19.496172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=18d6f0253aa44f069db625dde517fb18)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:19.498200 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:19.500909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:19.556396 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18d6f0253aa44f069db625dde517fb18'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:19.557056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18d6f0253aa44f069db625dde517fb18'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:19.564207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18d6f0253aa44f069db625dde517fb18'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:19.576406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18d6f0253aa44f069db625dde517fb18'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:19.577134 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18d6f0253aa44f069db625dde517fb18'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:19.589959 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1275/2565] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:19 2022] DELETE /identity/v3/users/18d6f0253aa44f069db625dde517fb18 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:19.595343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4663d-5fbb-4ee9-8da8-f6333184d021 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:19.595811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4663d-5fbb-4ee9-8da8-f6333184d021 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:19.595934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4663d-5fbb-4ee9-8da8-f6333184d021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:19.614220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8de4663d-5fbb-4ee9-8da8-f6333184d021 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:19.615496 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8de4663d-5fbb-4ee9-8da8-f6333184d021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:19.673723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1291/2566] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:19.683169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:19.684543 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:19.743633 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc38db-a345-45b3-8b3d-7485d4f6dcfb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:19.747297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0989400-b0f0-44ba-bee3-1691c0c4e1c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:19.747963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0989400-b0f0-44ba-bee3-1691c0c4e1c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:19.748229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0989400-b0f0-44ba-bee3-1691c0c4e1c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:19.748508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0989400-b0f0-44ba-bee3-1691c0c4e1c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:19.757598 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0989400-b0f0-44ba-bee3-1691c0c4e1c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:19.759061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0989400-b0f0-44ba-bee3-1691c0c4e1c4 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:19.769037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1276/2567] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:20.034606 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4663d-5fbb-4ee9-8da8-f6333184d021 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:20.036175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8de4663d-5fbb-4ee9-8da8-f6333184d021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:20.052122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4663d-5fbb-4ee9-8da8-f6333184d021 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:20.057759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:20.058318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:20.058602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:20.058883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] PATH_INFO: `/v3/projects/861a750092db47f8868e4bef039adc28` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:20.060190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=861a750092db47f8868e4bef039adc28)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:20.062659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:20.116463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '861a750092db47f8868e4bef039adc28', 'request_id': 'req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.125541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '861a750092db47f8868e4bef039adc28', 'request_id': 'req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.132075 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '861a750092db47f8868e4bef039adc28', 'request_id': 'req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.143372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] The token cache is being invalidate because project 861a750092db47f8868e4bef039adc28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:20.143651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.144297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.145822 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1292/2568] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:20 2022] DELETE /identity/v3/projects/861a750092db47f8868e4bef039adc28 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:20.149836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0989400-b0f0-44ba-bee3-1691c0c4e1c4 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:20.159884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0989400-b0f0-44ba-bee3-1691c0c4e1c4 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:20.222221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa0396-7044-43ba-8e5c-c38c843ede95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:20.223092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa0396-7044-43ba-8e5c-c38c843ede95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:20.223373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa0396-7044-43ba-8e5c-c38c843ede95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:20.223648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa0396-7044-43ba-8e5c-c38c843ede95 admin admin] PATH_INFO: `/v3/users/f74a22c40d1c40aa808c84381c2fa355` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:20.224826 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa0396-7044-43ba-8e5c-c38c843ede95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f74a22c40d1c40aa808c84381c2fa355)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:20.226771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa0396-7044-43ba-8e5c-c38c843ede95 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:20.254451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:20.275578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29aa0396-7044-43ba-8e5c-c38c843ede95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f74a22c40d1c40aa808c84381c2fa355'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.276414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29aa0396-7044-43ba-8e5c-c38c843ede95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f74a22c40d1c40aa808c84381c2fa355'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.286002 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29aa0396-7044-43ba-8e5c-c38c843ede95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f74a22c40d1c40aa808c84381c2fa355'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.298848 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29aa0396-7044-43ba-8e5c-c38c843ede95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f74a22c40d1c40aa808c84381c2fa355'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.299568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29aa0396-7044-43ba-8e5c-c38c843ede95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f74a22c40d1c40aa808c84381c2fa355'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.311749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1277/2569] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:20 2022] DELETE /identity/v3/users/f74a22c40d1c40aa808c84381c2fa355 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:20.580541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:20.582567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:20.598106 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93d170-5fd9-4c42-89ca-e96e9c4c48e6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:20.625622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:20.663446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:20.664326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:20.664611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:20.664890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] PATH_INFO: `/v3/projects/8361558c32f64b7fa2b9b7644fa4c3b5` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:20.666108 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8361558c32f64b7fa2b9b7644fa4c3b5)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:20.668045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:20.714575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8361558c32f64b7fa2b9b7644fa4c3b5', 'request_id': 'req-145187d6-202c-4693-b5b9-2b93f3c085e8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.722467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8361558c32f64b7fa2b9b7644fa4c3b5', 'request_id': 'req-145187d6-202c-4693-b5b9-2b93f3c085e8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.728723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8361558c32f64b7fa2b9b7644fa4c3b5', 'request_id': 'req-145187d6-202c-4693-b5b9-2b93f3c085e8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.738957 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] The token cache is being invalidate because project 8361558c32f64b7fa2b9b7644fa4c3b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:20.739244 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.739876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-145187d6-202c-4693-b5b9-2b93f3c085e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:20.741390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1293/2570] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:20 2022] DELETE /identity/v3/projects/8361558c32f64b7fa2b9b7644fa4c3b5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:20.746297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca498b47-e2ad-40ed-a3eb-d75a93996e2a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:20.746815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca498b47-e2ad-40ed-a3eb-d75a93996e2a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:20.746912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca498b47-e2ad-40ed-a3eb-d75a93996e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:20.765052 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ca498b47-e2ad-40ed-a3eb-d75a93996e2a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:20.767552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ca498b47-e2ad-40ed-a3eb-d75a93996e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:20.826398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1278/2571] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:20.835887 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a4392-494e-4ea6-bc1f-9bb506623cb4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:20.836180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a4392-494e-4ea6-bc1f-9bb506623cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:20.836462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a4392-494e-4ea6-bc1f-9bb506623cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:20.854692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d47a4392-494e-4ea6-bc1f-9bb506623cb4 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:20.855880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d47a4392-494e-4ea6-bc1f-9bb506623cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:20.864378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1294/2572] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:20.868606 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca498b47-e2ad-40ed-a3eb-d75a93996e2a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:20.878977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca498b47-e2ad-40ed-a3eb-d75a93996e2a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:20.883467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0ef69-f187-435f-b176-760c31eb9d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:20.884118 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0ef69-f187-435f-b176-760c31eb9d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:20.884401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0ef69-f187-435f-b176-760c31eb9d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:20.884677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0ef69-f187-435f-b176-760c31eb9d0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:20.885429 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0ef69-f187-435f-b176-760c31eb9d0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:20.885761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0ef69-f187-435f-b176-760c31eb9d0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:20.887398 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0ef69-f187-435f-b176-760c31eb9d0d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:20.899825 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1279/2573] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:20.904242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a4392-494e-4ea6-bc1f-9bb506623cb4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:20.913720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a4392-494e-4ea6-bc1f-9bb506623cb4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:20.919899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647222-7d87-41d8-b62d-bcbf48eedb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:20.920498 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647222-7d87-41d8-b62d-bcbf48eedb77 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:20.920785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647222-7d87-41d8-b62d-bcbf48eedb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:20.921065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647222-7d87-41d8-b62d-bcbf48eedb77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:20.921789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f647222-7d87-41d8-b62d-bcbf48eedb77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:20.923878 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f647222-7d87-41d8-b62d-bcbf48eedb77 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:20.949276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1295/2574] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:20 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:20.953519 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0ef69-f187-435f-b176-760c31eb9d0d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:20.964209 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0ef69-f187-435f-b176-760c31eb9d0d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:21.027883 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9363f5f-71e3-429e-903a-06215f55352a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:21.028566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9363f5f-71e3-429e-903a-06215f55352a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.028850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9363f5f-71e3-429e-903a-06215f55352a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.029128 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9363f5f-71e3-429e-903a-06215f55352a admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.029828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9363f5f-71e3-429e-903a-06215f55352a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:21.032113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9363f5f-71e3-429e-903a-06215f55352a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:21.063433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1280/2575] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:20 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:21.067464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647222-7d87-41d8-b62d-bcbf48eedb77 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:21.079135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647222-7d87-41d8-b62d-bcbf48eedb77 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:21.083816 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f441b-e615-40fd-9895-639e806cc66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:21.084484 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f441b-e615-40fd-9895-639e806cc66d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.084771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f441b-e615-40fd-9895-639e806cc66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.085079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f441b-e615-40fd-9895-639e806cc66d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.085808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f441b-e615-40fd-9895-639e806cc66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:21.087433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f441b-e615-40fd-9895-639e806cc66d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:21.098073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1296/2576] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:21.101968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9363f5f-71e3-429e-903a-06215f55352a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:21.113054 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9363f5f-71e3-429e-903a-06215f55352a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:21.117827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b74dcd-db24-46cb-b497-dc26b0fc81f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:21.118579 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b74dcd-db24-46cb-b497-dc26b0fc81f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.118873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b74dcd-db24-46cb-b497-dc26b0fc81f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.119159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b74dcd-db24-46cb-b497-dc26b0fc81f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.119898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b74dcd-db24-46cb-b497-dc26b0fc81f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:21.121501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b74dcd-db24-46cb-b497-dc26b0fc81f1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:21.136761 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1281/2577] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:21.139134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f441b-e615-40fd-9895-639e806cc66d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:21.153585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f441b-e615-40fd-9895-639e806cc66d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:21.158448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a5206-b44f-461c-8742-37b429e3068d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:21.159336 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a5206-b44f-461c-8742-37b429e3068d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.159622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a5206-b44f-461c-8742-37b429e3068d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.159900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a5206-b44f-461c-8742-37b429e3068d admin admin] PATH_INFO: `/v3/projects/99271631dace4268b358ae21988d0817/users/4f9053c0dfe34f77af18cb70e23fdc14/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.181179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a5206-b44f-461c-8742-37b429e3068d admin admin] RBAC: Authorizing `identity:create_grant(project_id=99271631dace4268b358ae21988d0817, user_id=4f9053c0dfe34f77af18cb70e23fdc14, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:21.183967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a5206-b44f-461c-8742-37b429e3068d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:21.200450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1297/2578] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:21 2022] PUT /identity/v3/projects/99271631dace4268b358ae21988d0817/users/4f9053c0dfe34f77af18cb70e23fdc14/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:21.211129 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595f2a3-9346-435b-ac57-025b96155e06 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.211422 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595f2a3-9346-435b-ac57-025b96155e06 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.211705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595f2a3-9346-435b-ac57-025b96155e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.242338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:21.259763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5595f2a3-9346-435b-ac57-025b96155e06 None None] MFA Rules not processed for user `4f9053c0dfe34f77af18cb70e23fdc14`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:21.261054 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5595f2a3-9346-435b-ac57-025b96155e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:21.339850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1282/2579] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:21.345284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a5206-b44f-461c-8742-37b429e3068d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:21.406796 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:21.418497 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a5206-b44f-461c-8742-37b429e3068d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:21.424141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc47750-3e0d-4692-9039-bd9c0ef6e4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:21.424802 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc47750-3e0d-4692-9039-bd9c0ef6e4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.425088 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc47750-3e0d-4692-9039-bd9c0ef6e4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.425369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc47750-3e0d-4692-9039-bd9c0ef6e4c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.426097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc47750-3e0d-4692-9039-bd9c0ef6e4c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:21.428043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc47750-3e0d-4692-9039-bd9c0ef6e4c7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:21.453166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1298/2580] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:21 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:21.457381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595f2a3-9346-435b-ac57-025b96155e06 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:21.468017 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595f2a3-9346-435b-ac57-025b96155e06 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:21.525451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8003e-eb96-4916-81d4-500bd1dd63a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:21.526159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8003e-eb96-4916-81d4-500bd1dd63a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.526440 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8003e-eb96-4916-81d4-500bd1dd63a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.526713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8003e-eb96-4916-81d4-500bd1dd63a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.527669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8003e-eb96-4916-81d4-500bd1dd63a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:21.529614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8003e-eb96-4916-81d4-500bd1dd63a5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:21.550818 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1283/2581] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:21 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:21.554873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc47750-3e0d-4692-9039-bd9c0ef6e4c7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:21.565386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc47750-3e0d-4692-9039-bd9c0ef6e4c7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:21.570001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267ef5-0123-44af-ba96-0f319ce85b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:21.570658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267ef5-0123-44af-ba96-0f319ce85b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.570937 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267ef5-0123-44af-ba96-0f319ce85b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.571212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267ef5-0123-44af-ba96-0f319ce85b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.571938 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48267ef5-0123-44af-ba96-0f319ce85b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:21.573514 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48267ef5-0123-44af-ba96-0f319ce85b0b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:21.584220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1299/2582] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:21.588314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8003e-eb96-4916-81d4-500bd1dd63a5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:21.599424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8003e-eb96-4916-81d4-500bd1dd63a5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:21.606457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbf6b3-564c-4286-a2e7-ed77b9a9f871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:21.607335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbf6b3-564c-4286-a2e7-ed77b9a9f871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.607708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbf6b3-564c-4286-a2e7-ed77b9a9f871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.608057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbf6b3-564c-4286-a2e7-ed77b9a9f871 admin admin] PATH_INFO: `/v3/projects/f564cef729ae4e96a3cf2cb534f3f548/users/175723b42c9e42faa475accafc492a38/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.624966 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbbf6b3-564c-4286-a2e7-ed77b9a9f871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f564cef729ae4e96a3cf2cb534f3f548, user_id=175723b42c9e42faa475accafc492a38, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:21.628031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbbf6b3-564c-4286-a2e7-ed77b9a9f871 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:21.641994 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1284/2583] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:21 2022] PUT /identity/v3/projects/f564cef729ae4e96a3cf2cb534f3f548/users/175723b42c9e42faa475accafc492a38/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:21.646688 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267ef5-0123-44af-ba96-0f319ce85b0b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:21.658146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267ef5-0123-44af-ba96-0f319ce85b0b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:21.721904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c20dc8-7c2a-4a89-af95-9440dd97958e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:21.722610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c20dc8-7c2a-4a89-af95-9440dd97958e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.722893 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c20dc8-7c2a-4a89-af95-9440dd97958e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.723169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c20dc8-7c2a-4a89-af95-9440dd97958e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.723914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c20dc8-7c2a-4a89-af95-9440dd97958e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:21.725561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c20dc8-7c2a-4a89-af95-9440dd97958e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:21.737632 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1300/2584] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:21.741812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbf6b3-564c-4286-a2e7-ed77b9a9f871 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:21.752800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbf6b3-564c-4286-a2e7-ed77b9a9f871 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:21.757511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbbb87-dfef-49af-b467-5def1a9269ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:21.758448 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbbb87-dfef-49af-b467-5def1a9269ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.758712 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbbb87-dfef-49af-b467-5def1a9269ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.759007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbbb87-dfef-49af-b467-5def1a9269ca admin admin] PATH_INFO: `/v3/projects/f564cef729ae4e96a3cf2cb534f3f548/users/175723b42c9e42faa475accafc492a38/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.760987 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbbb87-dfef-49af-b467-5def1a9269ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f564cef729ae4e96a3cf2cb534f3f548, user_id=175723b42c9e42faa475accafc492a38, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:21.763643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbbb87-dfef-49af-b467-5def1a9269ca admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:21.774864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1285/2585] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:21 2022] PUT /identity/v3/projects/f564cef729ae4e96a3cf2cb534f3f548/users/175723b42c9e42faa475accafc492a38/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:21.786530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97090df4-2d80-4107-aaaf-e08b2eb706ed None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.786832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97090df4-2d80-4107-aaaf-e08b2eb706ed None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.787118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97090df4-2d80-4107-aaaf-e08b2eb706ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.815488 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-97090df4-2d80-4107-aaaf-e08b2eb706ed None None] MFA Rules not processed for user `175723b42c9e42faa475accafc492a38`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:21.816696 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-97090df4-2d80-4107-aaaf-e08b2eb706ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:21.891804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1301/2586] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:21.903649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbbb87-dfef-49af-b467-5def1a9269ca admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:21.905708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-becbbb87-dfef-49af-b467-5def1a9269ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:21.967592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbbb87-dfef-49af-b467-5def1a9269ca admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:21.971508 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80992368-1fd0-4107-ad7e-2122e7676def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:21.972182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80992368-1fd0-4107-ad7e-2122e7676def service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:21.972466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80992368-1fd0-4107-ad7e-2122e7676def service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:21.972746 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80992368-1fd0-4107-ad7e-2122e7676def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:21.978906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:21.983691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80992368-1fd0-4107-ad7e-2122e7676def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:21.986009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80992368-1fd0-4107-ad7e-2122e7676def service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:21.995971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1286/2587] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:22.080823 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97090df4-2d80-4107-aaaf-e08b2eb706ed None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:22.091162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97090df4-2d80-4107-aaaf-e08b2eb706ed None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:22.095752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4198c8c-9c1f-4ff5-981f-5bd4177db42d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:22.096732 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4198c8c-9c1f-4ff5-981f-5bd4177db42d service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:22.096884 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4198c8c-9c1f-4ff5-981f-5bd4177db42d service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:22.097054 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4198c8c-9c1f-4ff5-981f-5bd4177db42d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:22.149888 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:22.164206 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4198c8c-9c1f-4ff5-981f-5bd4177db42d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:22.165787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4198c8c-9c1f-4ff5-981f-5bd4177db42d service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:22.175626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1302/2588] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:27.629647 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80992368-1fd0-4107-ad7e-2122e7676def service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:27.631639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-80992368-1fd0-4107-ad7e-2122e7676def service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:27.699231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80992368-1fd0-4107-ad7e-2122e7676def service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:27.703488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c996d-d940-4ae0-b5a8-bbc03546af98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:27.704470 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c996d-d940-4ae0-b5a8-bbc03546af98 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:27.704894 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c996d-d940-4ae0-b5a8-bbc03546af98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:27.705303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c996d-d940-4ae0-b5a8-bbc03546af98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:27.707374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f12c996d-d940-4ae0-b5a8-bbc03546af98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:27.781632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c996d-d940-4ae0-b5a8-bbc03546af98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:27.783131 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c996d-d940-4ae0-b5a8-bbc03546af98 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:27.792952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1287/2589] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:27.970790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4198c8c-9c1f-4ff5-981f-5bd4177db42d service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:27.982226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4198c8c-9c1f-4ff5-981f-5bd4177db42d service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:27.987917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:27.988923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:27.989273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:27.989621 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] PATH_INFO: `/v3/users/4f9053c0dfe34f77af18cb70e23fdc14` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:27.990992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f9053c0dfe34f77af18cb70e23fdc14)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:27.992999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:28.045779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f9053c0dfe34f77af18cb70e23fdc14'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.061420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f9053c0dfe34f77af18cb70e23fdc14'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.068898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f9053c0dfe34f77af18cb70e23fdc14'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.069648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f9053c0dfe34f77af18cb70e23fdc14'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.070525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f9053c0dfe34f77af18cb70e23fdc14'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.084193 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1303/2590] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:27 2022] DELETE /identity/v3/users/4f9053c0dfe34f77af18cb70e23fdc14 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:28.088989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004d80c-6aa7-41b4-a7d9-7a04f94d265f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:28.089494 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004d80c-6aa7-41b4-a7d9-7a04f94d265f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:28.089787 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004d80c-6aa7-41b4-a7d9-7a04f94d265f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:28.108349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3004d80c-6aa7-41b4-a7d9-7a04f94d265f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:28.109759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3004d80c-6aa7-41b4-a7d9-7a04f94d265f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:28.172284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1288/2591] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:28.238900 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:28.240403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:28.299903 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db863f7-97b9-41ab-b1da-8e8ebb120adf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:28.303500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e615c64-7a2f-4fa7-9f45-815dc1719e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:28.304148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e615c64-7a2f-4fa7-9f45-815dc1719e7e service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:28.304427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e615c64-7a2f-4fa7-9f45-815dc1719e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:28.304700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e615c64-7a2f-4fa7-9f45-815dc1719e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:28.315017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e615c64-7a2f-4fa7-9f45-815dc1719e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:28.316445 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e615c64-7a2f-4fa7-9f45-815dc1719e7e service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:28.326494 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1304/2592] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:28.613570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004d80c-6aa7-41b4-a7d9-7a04f94d265f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:28.616057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3004d80c-6aa7-41b4-a7d9-7a04f94d265f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:28.633351 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004d80c-6aa7-41b4-a7d9-7a04f94d265f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:28.642737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:28.643922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:28.644405 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:28.644938 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] PATH_INFO: `/v3/projects/99271631dace4268b358ae21988d0817` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:28.646623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] RBAC: Authorizing `identity:delete_project(project_id=99271631dace4268b358ae21988d0817)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:28.649124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:28.701802 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:28.716941 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99271631dace4268b358ae21988d0817', 'request_id': 'req-3c708444-b74f-43c0-b7e9-c4d79a98d68d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.726334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99271631dace4268b358ae21988d0817', 'request_id': 'req-3c708444-b74f-43c0-b7e9-c4d79a98d68d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.735456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99271631dace4268b358ae21988d0817', 'request_id': 'req-3c708444-b74f-43c0-b7e9-c4d79a98d68d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.750423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] The token cache is being invalidate because project 99271631dace4268b358ae21988d0817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:28.750970 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.751710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.753982 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1289/2593] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:28 2022] DELETE /identity/v3/projects/99271631dace4268b358ae21988d0817 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:28.758238 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e615c64-7a2f-4fa7-9f45-815dc1719e7e service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:28.770817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e615c64-7a2f-4fa7-9f45-815dc1719e7e service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:28.839042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4b814-6621-441a-b3fe-7e49cc3fb7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:28.839924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4b814-6621-441a-b3fe-7e49cc3fb7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:28.840207 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4b814-6621-441a-b3fe-7e49cc3fb7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:28.840480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4b814-6621-441a-b3fe-7e49cc3fb7bd admin admin] PATH_INFO: `/v3/users/175723b42c9e42faa475accafc492a38` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:28.841689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4b814-6621-441a-b3fe-7e49cc3fb7bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=175723b42c9e42faa475accafc492a38)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:28.844873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4b814-6621-441a-b3fe-7e49cc3fb7bd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:28.847534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:28.898142 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-48e4b814-6621-441a-b3fe-7e49cc3fb7bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '175723b42c9e42faa475accafc492a38'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.911442 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-48e4b814-6621-441a-b3fe-7e49cc3fb7bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '175723b42c9e42faa475accafc492a38'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.920536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-48e4b814-6621-441a-b3fe-7e49cc3fb7bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '175723b42c9e42faa475accafc492a38'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.921350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-48e4b814-6621-441a-b3fe-7e49cc3fb7bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '175723b42c9e42faa475accafc492a38'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.922626 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-48e4b814-6621-441a-b3fe-7e49cc3fb7bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '175723b42c9e42faa475accafc492a38'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:28.936500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1305/2594] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:28 2022] DELETE /identity/v3/users/175723b42c9e42faa475accafc492a38 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:29.300858 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:29.305506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:29.335164 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c708444-b74f-43c0-b7e9-c4d79a98d68d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:29.414268 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:29.415236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:29.415588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:29.415935 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] PATH_INFO: `/v3/projects/f564cef729ae4e96a3cf2cb534f3f548` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:29.417250 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f564cef729ae4e96a3cf2cb534f3f548)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:29.419359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:29.482322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f564cef729ae4e96a3cf2cb534f3f548', 'request_id': 'req-28392016-7d17-427a-a485-4d4a9c8d2fa2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:29.489596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f564cef729ae4e96a3cf2cb534f3f548', 'request_id': 'req-28392016-7d17-427a-a485-4d4a9c8d2fa2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:29.497703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f564cef729ae4e96a3cf2cb534f3f548', 'request_id': 'req-28392016-7d17-427a-a485-4d4a9c8d2fa2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:29.512702 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] The token cache is being invalidate because project f564cef729ae4e96a3cf2cb534f3f548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:29.513088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:29.513791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-28392016-7d17-427a-a485-4d4a9c8d2fa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:29.518160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1290/2595] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:29 2022] DELETE /identity/v3/projects/f564cef729ae4e96a3cf2cb534f3f548 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:29.521181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca7a9-31df-4b25-b139-7cecd72a5922 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:29.521672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca7a9-31df-4b25-b139-7cecd72a5922 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:29.521776 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca7a9-31df-4b25-b139-7cecd72a5922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:29.543730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-420ca7a9-31df-4b25-b139-7cecd72a5922 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:29.544939 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-420ca7a9-31df-4b25-b139-7cecd72a5922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:29.624680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1306/2596] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:29.626917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4a478-1c4f-44fb-a50c-1139032f38ec None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:29.627214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4a478-1c4f-44fb-a50c-1139032f38ec None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:29.627499 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4a478-1c4f-44fb-a50c-1139032f38ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:29.646261 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-03b4a478-1c4f-44fb-a50c-1139032f38ec None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:29.647469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-03b4a478-1c4f-44fb-a50c-1139032f38ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:29.663452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1291/2597] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:29.664263 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ca7a9-31df-4b25-b139-7cecd72a5922 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:29.675990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ca7a9-31df-4b25-b139-7cecd72a5922 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:29.680534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40259771-c6a3-4235-8df6-d29f3dfeeab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:29.681205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40259771-c6a3-4235-8df6-d29f3dfeeab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:29.681522 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40259771-c6a3-4235-8df6-d29f3dfeeab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:29.681803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40259771-c6a3-4235-8df6-d29f3dfeeab2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:29.682574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40259771-c6a3-4235-8df6-d29f3dfeeab2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:29.682882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40259771-c6a3-4235-8df6-d29f3dfeeab2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:29.684552 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40259771-c6a3-4235-8df6-d29f3dfeeab2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:29.697065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1307/2598] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:29.701427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4a478-1c4f-44fb-a50c-1139032f38ec None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:29.711539 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4a478-1c4f-44fb-a50c-1139032f38ec None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:29.717871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b6c6a-8f0d-4b44-ae0b-8e02e9ac8b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:29.719381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6c6a-8f0d-4b44-ae0b-8e02e9ac8b85 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:29.719728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6c6a-8f0d-4b44-ae0b-8e02e9ac8b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:29.720083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6c6a-8f0d-4b44-ae0b-8e02e9ac8b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:29.720951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b6c6a-8f0d-4b44-ae0b-8e02e9ac8b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:29.723141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b6c6a-8f0d-4b44-ae0b-8e02e9ac8b85 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:29.771561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1292/2599] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:29 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:29.775154 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40259771-c6a3-4235-8df6-d29f3dfeeab2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:29.793813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40259771-c6a3-4235-8df6-d29f3dfeeab2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:29.901759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096a91d-6116-447b-9de1-1d98162f0586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:29.903135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a91d-6116-447b-9de1-1d98162f0586 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:29.903689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a91d-6116-447b-9de1-1d98162f0586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:29.903689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a91d-6116-447b-9de1-1d98162f0586 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:29.904363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096a91d-6116-447b-9de1-1d98162f0586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:29.908245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096a91d-6116-447b-9de1-1d98162f0586 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:29.936111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:29.951987 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1308/2600] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:29 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:38:29.954329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b6c6a-8f0d-4b44-ae0b-8e02e9ac8b85 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:29.965961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b6c6a-8f0d-4b44-ae0b-8e02e9ac8b85 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:29.972120 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18eb0e5-bd3a-45f6-9aee-831999daab69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:29.973056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18eb0e5-bd3a-45f6-9aee-831999daab69 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:29.973412 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18eb0e5-bd3a-45f6-9aee-831999daab69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:29.973757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18eb0e5-bd3a-45f6-9aee-831999daab69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:29.974730 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18eb0e5-bd3a-45f6-9aee-831999daab69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:29.976511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18eb0e5-bd3a-45f6-9aee-831999daab69 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:29.992812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1293/2601] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:29.994928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096a91d-6116-447b-9de1-1d98162f0586 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:30.007806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096a91d-6116-447b-9de1-1d98162f0586 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:30.013180 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8acef-fdef-4976-8b5c-4b40d10bf4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:30.014017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8acef-fdef-4976-8b5c-4b40d10bf4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:30.014535 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8acef-fdef-4976-8b5c-4b40d10bf4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:30.014892 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8acef-fdef-4976-8b5c-4b40d10bf4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:30.016745 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8acef-fdef-4976-8b5c-4b40d10bf4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:30.018594 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8acef-fdef-4976-8b5c-4b40d10bf4c5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:30.052267 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1309/2602] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:30.053433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18eb0e5-bd3a-45f6-9aee-831999daab69 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:30.065759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18eb0e5-bd3a-45f6-9aee-831999daab69 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:30.072638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f9a2c-9a21-4b6f-92e3-99ced61de7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:30.073755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f9a2c-9a21-4b6f-92e3-99ced61de7a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:30.075191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f9a2c-9a21-4b6f-92e3-99ced61de7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:30.075501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f9a2c-9a21-4b6f-92e3-99ced61de7a4 admin admin] PATH_INFO: `/v3/projects/12370a7753c14f6082fadd2fb59bc65a/users/d41470dd2f7849949fba9d45be404bf0/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:30.100849 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f9a2c-9a21-4b6f-92e3-99ced61de7a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12370a7753c14f6082fadd2fb59bc65a, user_id=d41470dd2f7849949fba9d45be404bf0, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:30.103913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f9a2c-9a21-4b6f-92e3-99ced61de7a4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:30.106498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:30.130821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1294/2603] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:30 2022] PUT /identity/v3/projects/12370a7753c14f6082fadd2fb59bc65a/users/d41470dd2f7849949fba9d45be404bf0/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:30.142101 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1c35-7616-48b2-ae03-9645c2a23090 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:30.142580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1c35-7616-48b2-ae03-9645c2a23090 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:30.143058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1c35-7616-48b2-ae03-9645c2a23090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:30.178663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e98e1c35-7616-48b2-ae03-9645c2a23090 None None] MFA Rules not processed for user `d41470dd2f7849949fba9d45be404bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:30.180044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e98e1c35-7616-48b2-ae03-9645c2a23090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:30.263532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1310/2604] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:30.268541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f9a2c-9a21-4b6f-92e3-99ced61de7a4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:30.377271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f9a2c-9a21-4b6f-92e3-99ced61de7a4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:30.385301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfd6bb-c20d-4776-86ab-b72bafba9284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:30.386064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfd6bb-c20d-4776-86ab-b72bafba9284 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:30.386372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfd6bb-c20d-4776-86ab-b72bafba9284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:30.386632 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfd6bb-c20d-4776-86ab-b72bafba9284 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:30.387338 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfd6bb-c20d-4776-86ab-b72bafba9284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:30.389280 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfd6bb-c20d-4776-86ab-b72bafba9284 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:30.417684 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1295/2605] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:30 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:30.421848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1c35-7616-48b2-ae03-9645c2a23090 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:30.433166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1c35-7616-48b2-ae03-9645c2a23090 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:30.495100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1d7db-c9a3-4571-ad4f-613d8e11512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:30.495704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1d7db-c9a3-4571-ad4f-613d8e11512f admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:30.495990 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1d7db-c9a3-4571-ad4f-613d8e11512f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:30.496269 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1d7db-c9a3-4571-ad4f-613d8e11512f admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:30.496986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1d7db-c9a3-4571-ad4f-613d8e11512f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:30.498938 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1d7db-c9a3-4571-ad4f-613d8e11512f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:30.521086 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1311/2606] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:30 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:30.524907 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfd6bb-c20d-4776-86ab-b72bafba9284 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:30.535798 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfd6bb-c20d-4776-86ab-b72bafba9284 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:30.540531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bee51-363a-4a09-acbd-c5ae22734562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:30.541476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bee51-363a-4a09-acbd-c5ae22734562 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:30.541770 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bee51-363a-4a09-acbd-c5ae22734562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:30.542183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bee51-363a-4a09-acbd-c5ae22734562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:30.542990 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bee51-363a-4a09-acbd-c5ae22734562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:30.544669 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bee51-363a-4a09-acbd-c5ae22734562 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:30.556455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1296/2607] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:30.560724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1d7db-c9a3-4571-ad4f-613d8e11512f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:30.571376 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1d7db-c9a3-4571-ad4f-613d8e11512f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:30.576325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffa2af-3e02-4e1c-990f-04b48c54f432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:30.577378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffa2af-3e02-4e1c-990f-04b48c54f432 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:30.577844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffa2af-3e02-4e1c-990f-04b48c54f432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:30.578797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffa2af-3e02-4e1c-990f-04b48c54f432 admin admin] PATH_INFO: `/v3/projects/8f0aa35b2d4f416688d88754a5be87f9/users/a7a842043fcf4bcd9cbafaf43742d782/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:30.594672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffa2af-3e02-4e1c-990f-04b48c54f432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0aa35b2d4f416688d88754a5be87f9, user_id=a7a842043fcf4bcd9cbafaf43742d782, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:30.597544 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffa2af-3e02-4e1c-990f-04b48c54f432 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:30.613378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1312/2608] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:30 2022] PUT /identity/v3/projects/8f0aa35b2d4f416688d88754a5be87f9/users/a7a842043fcf4bcd9cbafaf43742d782/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:30.617375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bee51-363a-4a09-acbd-c5ae22734562 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:30.630601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bee51-363a-4a09-acbd-c5ae22734562 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:30.705657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4c6a7-88af-4c6d-835e-590e8fc38a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:30.706426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4c6a7-88af-4c6d-835e-590e8fc38a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:30.706710 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4c6a7-88af-4c6d-835e-590e8fc38a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:30.706924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4c6a7-88af-4c6d-835e-590e8fc38a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:30.708974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4c6a7-88af-4c6d-835e-590e8fc38a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:30.709422 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4c6a7-88af-4c6d-835e-590e8fc38a6c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:30.722116 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1297/2609] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:30.726179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffa2af-3e02-4e1c-990f-04b48c54f432 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:30.742308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffa2af-3e02-4e1c-990f-04b48c54f432 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:30.752685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f350859-7fc4-41d5-acc5-c569d3539de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:30.755328 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f350859-7fc4-41d5-acc5-c569d3539de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:30.755622 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f350859-7fc4-41d5-acc5-c569d3539de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:30.755908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f350859-7fc4-41d5-acc5-c569d3539de8 admin admin] PATH_INFO: `/v3/projects/8f0aa35b2d4f416688d88754a5be87f9/users/a7a842043fcf4bcd9cbafaf43742d782/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:30.758040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f350859-7fc4-41d5-acc5-c569d3539de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0aa35b2d4f416688d88754a5be87f9, user_id=a7a842043fcf4bcd9cbafaf43742d782, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:30.760734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f350859-7fc4-41d5-acc5-c569d3539de8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:30.772459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1313/2610] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:30 2022] PUT /identity/v3/projects/8f0aa35b2d4f416688d88754a5be87f9/users/a7a842043fcf4bcd9cbafaf43742d782/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:30.784875 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457b17a-dfd2-4f99-901d-8c26efc72467 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:30.785173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457b17a-dfd2-4f99-901d-8c26efc72467 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:30.785458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457b17a-dfd2-4f99-901d-8c26efc72467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:30.816940 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a457b17a-dfd2-4f99-901d-8c26efc72467 None None] MFA Rules not processed for user `a7a842043fcf4bcd9cbafaf43742d782`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:30.818239 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a457b17a-dfd2-4f99-901d-8c26efc72467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:30.875725 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:30.896750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1298/2611] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:30.908052 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f350859-7fc4-41d5-acc5-c569d3539de8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:30.920131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6f350859-7fc4-41d5-acc5-c569d3539de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:30.977736 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:31.013117 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f350859-7fc4-41d5-acc5-c569d3539de8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:31.019601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a365c99-933d-4370-baab-f8a7cf9c3892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:31.020390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a365c99-933d-4370-baab-f8a7cf9c3892 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:31.020689 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a365c99-933d-4370-baab-f8a7cf9c3892 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:31.021023 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a365c99-933d-4370-baab-f8a7cf9c3892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:31.042865 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a365c99-933d-4370-baab-f8a7cf9c3892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:31.044885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a365c99-933d-4370-baab-f8a7cf9c3892 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:31.065915 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1314/2612] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:31.346307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457b17a-dfd2-4f99-901d-8c26efc72467 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:31.357749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a457b17a-dfd2-4f99-901d-8c26efc72467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:31.529652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457b17a-dfd2-4f99-901d-8c26efc72467 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:31.538671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:31.539566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:31.539854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:31.540175 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] PATH_INFO: `/v3/users/a4990a985c4e45c4b1b07960d98f0923` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:31.541569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4990a985c4e45c4b1b07960d98f0923)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:31.545213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:31.614839 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4990a985c4e45c4b1b07960d98f0923'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:31.615577 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4990a985c4e45c4b1b07960d98f0923'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:31.634676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4990a985c4e45c4b1b07960d98f0923'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:31.654612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4990a985c4e45c4b1b07960d98f0923'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:31.655589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4990a985c4e45c4b1b07960d98f0923'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:31.672840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1299/2613] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:31 2022] DELETE /identity/v3/users/a4990a985c4e45c4b1b07960d98f0923 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:31.677340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18429b84-835c-4ed3-9e15-7e6fb3b8432a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:31.677929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18429b84-835c-4ed3-9e15-7e6fb3b8432a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:31.677929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18429b84-835c-4ed3-9e15-7e6fb3b8432a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:31.716010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-18429b84-835c-4ed3-9e15-7e6fb3b8432a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:31.717306 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-18429b84-835c-4ed3-9e15-7e6fb3b8432a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:31.733643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:31.733643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:31.845570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1315/2614] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:31.846610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18429b84-835c-4ed3-9e15-7e6fb3b8432a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:31.852660 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c289c-3bc1-462f-8757-c84778ef8779 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:31.857988 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:31.858129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-18429b84-835c-4ed3-9e15-7e6fb3b8432a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:31.862236 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:31.862821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:31.863200 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] PATH_INFO: `/v3/users/15ec5e26839f494e98ec70ac49ebe415` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:31.864953 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ec5e26839f494e98ec70ac49ebe415)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:31.867288 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:31.883303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:31.944777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ec5e26839f494e98ec70ac49ebe415'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:31.945498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ec5e26839f494e98ec70ac49ebe415'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:31.954311 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ec5e26839f494e98ec70ac49ebe415'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:31.970876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18429b84-835c-4ed3-9e15-7e6fb3b8432a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:31.972493 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ec5e26839f494e98ec70ac49ebe415'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:31.973263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ec5e26839f494e98ec70ac49ebe415'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:31.975148 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b59e6-0d4e-402d-9b34-fa302eaa566f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:31.975810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b59e6-0d4e-402d-9b34-fa302eaa566f service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:31.976093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b59e6-0d4e-402d-9b34-fa302eaa566f service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:31.976390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b59e6-0d4e-402d-9b34-fa302eaa566f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:31.978293 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a30b59e6-0d4e-402d-9b34-fa302eaa566f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:31.990404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1300/2615] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:31 2022] DELETE /identity/v3/users/15ec5e26839f494e98ec70ac49ebe415 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:31.991503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:31.993148 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:32.068174 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b59e6-0d4e-402d-9b34-fa302eaa566f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:32.069809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b59e6-0d4e-402d-9b34-fa302eaa566f service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:32.080246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1316/2616] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:32.081220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b59e6-0d4e-402d-9b34-fa302eaa566f service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:32.083470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a30b59e6-0d4e-402d-9b34-fa302eaa566f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:32.097712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:32.105182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee56a8a-32eb-41bc-8369-b08850dcd3b7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:32.108591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b59e6-0d4e-402d-9b34-fa302eaa566f service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:32.112376 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7e6d9-ac0c-4ed6-b435-013ee71f87a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:32.113079 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7e6d9-ac0c-4ed6-b435-013ee71f87a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.113363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7e6d9-ac0c-4ed6-b435-013ee71f87a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.113639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7e6d9-ac0c-4ed6-b435-013ee71f87a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.116130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-35c7e6d9-ac0c-4ed6-b435-013ee71f87a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:32.211879 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bfa82-9295-438e-8bef-a5d3072b2b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:32.213055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bfa82-9295-438e-8bef-a5d3072b2b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.213471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bfa82-9295-438e-8bef-a5d3072b2b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.213922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bfa82-9295-438e-8bef-a5d3072b2b24 admin admin] PATH_INFO: `/v3/users/d41470dd2f7849949fba9d45be404bf0` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.215427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bfa82-9295-438e-8bef-a5d3072b2b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41470dd2f7849949fba9d45be404bf0)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:32.217781 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bfa82-9295-438e-8bef-a5d3072b2b24 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:32.225344 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7e6d9-ac0c-4ed6-b435-013ee71f87a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:32.227231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7e6d9-ac0c-4ed6-b435-013ee71f87a8 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:32.246254 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1301/2617] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:32.247133 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7e6d9-ac0c-4ed6-b435-013ee71f87a8 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:32.248631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-35c7e6d9-ac0c-4ed6-b435-013ee71f87a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:32.262700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7e6d9-ac0c-4ed6-b435-013ee71f87a8 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:32.268544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:32.269505 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.269869 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.270263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] PATH_INFO: `/v3/projects/2be6e5aedba545f4b562423c55ddbf80` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.271566 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be6e5aedba545f4b562423c55ddbf80)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:32.273732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:32.282832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-251bfa82-9295-438e-8bef-a5d3072b2b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41470dd2f7849949fba9d45be404bf0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.302159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-251bfa82-9295-438e-8bef-a5d3072b2b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41470dd2f7849949fba9d45be404bf0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.316020 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-251bfa82-9295-438e-8bef-a5d3072b2b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41470dd2f7849949fba9d45be404bf0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.316897 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-251bfa82-9295-438e-8bef-a5d3072b2b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41470dd2f7849949fba9d45be404bf0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.317611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-251bfa82-9295-438e-8bef-a5d3072b2b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41470dd2f7849949fba9d45be404bf0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.324046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:32.335630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1317/2618] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:32 2022] DELETE /identity/v3/users/d41470dd2f7849949fba9d45be404bf0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:32.341153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88415357-9f6d-494a-bd34-e7de1e7e20bc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.341437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88415357-9f6d-494a-bd34-e7de1e7e20bc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.341751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88415357-9f6d-494a-bd34-e7de1e7e20bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.342501 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be6e5aedba545f4b562423c55ddbf80', 'request_id': 'req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.355013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be6e5aedba545f4b562423c55ddbf80', 'request_id': 'req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.362558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-88415357-9f6d-494a-bd34-e7de1e7e20bc None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:32.363313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be6e5aedba545f4b562423c55ddbf80', 'request_id': 'req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.363958 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-88415357-9f6d-494a-bd34-e7de1e7e20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:32.380157 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] The token cache is being invalidate because project 2be6e5aedba545f4b562423c55ddbf80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:32.380427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.381504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-742bd1dc-f6cc-4700-94a0-e6eb3d355f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.383325 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1302/2619] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:32 2022] DELETE /identity/v3/projects/2be6e5aedba545f4b562423c55ddbf80 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:32.393638 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea685f-e4ae-4dc1-a54e-9cfc17b08945 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.394278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea685f-e4ae-4dc1-a54e-9cfc17b08945 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.395896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea685f-e4ae-4dc1-a54e-9cfc17b08945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.418782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-82ea685f-e4ae-4dc1-a54e-9cfc17b08945 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:32.420204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-82ea685f-e4ae-4dc1-a54e-9cfc17b08945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:32.450682 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1318/2620] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:32.458179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88415357-9f6d-494a-bd34-e7de1e7e20bc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:32.460288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-88415357-9f6d-494a-bd34-e7de1e7e20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:32.488856 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1303/2621] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:32.496925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc39f29-ea66-43b9-a5d2-7668a8deb1a4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.497207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc39f29-ea66-43b9-a5d2-7668a8deb1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.497426 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:32.497574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc39f29-ea66-43b9-a5d2-7668a8deb1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.521924 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3fc39f29-ea66-43b9-a5d2-7668a8deb1a4 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:32.523466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3fc39f29-ea66-43b9-a5d2-7668a8deb1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:32.532426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1304/2622] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:32.542132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88415357-9f6d-494a-bd34-e7de1e7e20bc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:32.548185 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ee0a9-b4e5-4e26-8ee9-34e3d165e2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:32.548409 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc39f29-ea66-43b9-a5d2-7668a8deb1a4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:32.548916 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ee0a9-b4e5-4e26-8ee9-34e3d165e2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.549323 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ee0a9-b4e5-4e26-8ee9-34e3d165e2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.549674 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ee0a9-b4e5-4e26-8ee9-34e3d165e2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.563260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ee0a9-b4e5-4e26-8ee9-34e3d165e2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:32.564682 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ee0a9-b4e5-4e26-8ee9-34e3d165e2c2 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:32.568721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc39f29-ea66-43b9-a5d2-7668a8deb1a4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:32.573601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cd49d-bcb2-42c9-bdba-5911d393f5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:32.574306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cd49d-bcb2-42c9-bdba-5911d393f5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.574587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cd49d-bcb2-42c9-bdba-5911d393f5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.574865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cd49d-bcb2-42c9-bdba-5911d393f5eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.575596 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cd49d-bcb2-42c9-bdba-5911d393f5eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:32.575906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cd49d-bcb2-42c9-bdba-5911d393f5eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:32.577584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cd49d-bcb2-42c9-bdba-5911d393f5eb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:32.580341 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1319/2623] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:32.591313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1305/2624] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:32.596291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ee0a9-b4e5-4e26-8ee9-34e3d165e2c2 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:32.607469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ee0a9-b4e5-4e26-8ee9-34e3d165e2c2 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:32.614123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d4a34-9ae1-4c68-80e2-29f72f0b0e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:32.614874 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d4a34-9ae1-4c68-80e2-29f72f0b0e08 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.615284 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d4a34-9ae1-4c68-80e2-29f72f0b0e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.615750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d4a34-9ae1-4c68-80e2-29f72f0b0e08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.616524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d4a34-9ae1-4c68-80e2-29f72f0b0e08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:32.620377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d4a34-9ae1-4c68-80e2-29f72f0b0e08 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:32.650003 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1320/2625] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:32 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:32.653828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cd49d-bcb2-42c9-bdba-5911d393f5eb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:32.667531 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cd49d-bcb2-42c9-bdba-5911d393f5eb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:32.743626 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f11051-d924-4ff3-a0e6-b16255e44589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:32.744330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f11051-d924-4ff3-a0e6-b16255e44589 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.744593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f11051-d924-4ff3-a0e6-b16255e44589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.744876 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f11051-d924-4ff3-a0e6-b16255e44589 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.745583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f11051-d924-4ff3-a0e6-b16255e44589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:32.747720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f11051-d924-4ff3-a0e6-b16255e44589 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:32.773521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1306/2626] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:32 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:32.776135 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d4a34-9ae1-4c68-80e2-29f72f0b0e08 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:32.777975 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f11051-d924-4ff3-a0e6-b16255e44589 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:32.780336 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-31f11051-d924-4ff3-a0e6-b16255e44589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:32.793731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d4a34-9ae1-4c68-80e2-29f72f0b0e08 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:32.797788 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f11051-d924-4ff3-a0e6-b16255e44589 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:32.802734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b2ce1-8d07-4be7-9ee3-6e6ebe3624b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:32.803433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b2ce1-8d07-4be7-9ee3-6e6ebe3624b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.803720 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b2ce1-8d07-4be7-9ee3-6e6ebe3624b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.804000 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b2ce1-8d07-4be7-9ee3-6e6ebe3624b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.804742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b2ce1-8d07-4be7-9ee3-6e6ebe3624b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:32.805059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:32.806060 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.806418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.806774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] PATH_INFO: `/v3/projects/5a4681a07f154a268c06f7c26cf66db1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.806887 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b2ce1-8d07-4be7-9ee3-6e6ebe3624b3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:32.808124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4681a07f154a268c06f7c26cf66db1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:32.814295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:32.854260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1321/2627] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:32.858131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b2ce1-8d07-4be7-9ee3-6e6ebe3624b3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:32.862778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:32.877749 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b2ce1-8d07-4be7-9ee3-6e6ebe3624b3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:32.900198 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4681a07f154a268c06f7c26cf66db1', 'request_id': 'req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.910058 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4681a07f154a268c06f7c26cf66db1', 'request_id': 'req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.922360 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4681a07f154a268c06f7c26cf66db1', 'request_id': 'req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.945644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] The token cache is being invalidate because project 5a4681a07f154a268c06f7c26cf66db1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:32.946495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.948145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3740b1b-7a72-4b32-a9cf-0e6a94aef295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:32.951177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1307/2628] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:32 2022] DELETE /identity/v3/projects/5a4681a07f154a268c06f7c26cf66db1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:32.957033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb34e7-7070-4aa7-98f5-8f93a35e28a3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.957652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb34e7-7070-4aa7-98f5-8f93a35e28a3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.958183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb34e7-7070-4aa7-98f5-8f93a35e28a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.966618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a69668-26b2-4bde-9486-9a40eaed22f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:32.967454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a69668-26b2-4bde-9486-9a40eaed22f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:32.967748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a69668-26b2-4bde-9486-9a40eaed22f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:32.968452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a69668-26b2-4bde-9486-9a40eaed22f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:32.970623 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a69668-26b2-4bde-9486-9a40eaed22f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:32.972282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a69668-26b2-4bde-9486-9a40eaed22f6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:32.985354 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ebdb34e7-7070-4aa7-98f5-8f93a35e28a3 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:32.986581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1322/2629] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:32.987541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ebdb34e7-7070-4aa7-98f5-8f93a35e28a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:32.992800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a69668-26b2-4bde-9486-9a40eaed22f6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:32.995360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f5a69668-26b2-4bde-9486-9a40eaed22f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:33.005932 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1308/2630] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:33.007789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb34e7-7070-4aa7-98f5-8f93a35e28a3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:33.009212 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ebdb34e7-7070-4aa7-98f5-8f93a35e28a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:33.027192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a69668-26b2-4bde-9486-9a40eaed22f6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:33.028529 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb34e7-7070-4aa7-98f5-8f93a35e28a3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:33.033918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813c13a-0f00-4faf-b893-b98e550a6e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:33.035083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813c13a-0f00-4faf-b893-b98e550a6e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.035570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813c13a-0f00-4faf-b893-b98e550a6e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.035859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813c13a-0f00-4faf-b893-b98e550a6e22 admin admin] PATH_INFO: `/v3/projects/e707a56bc1944837b26f6306784d3012/users/6d37f54f6721419b8428487153d9a89c/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.047976 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:33.049204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.049731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.050256 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] PATH_INFO: `/v3/projects/12370a7753c14f6082fadd2fb59bc65a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.051770 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12370a7753c14f6082fadd2fb59bc65a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:33.054010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:33.065480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813c13a-0f00-4faf-b893-b98e550a6e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e707a56bc1944837b26f6306784d3012, user_id=6d37f54f6721419b8428487153d9a89c, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:33.068647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813c13a-0f00-4faf-b893-b98e550a6e22 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:33.084746 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1323/2631] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:32 2022] PUT /identity/v3/projects/e707a56bc1944837b26f6306784d3012/users/6d37f54f6721419b8428487153d9a89c/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:33.093618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f25d8-bcd5-41a9-8840-9131b8154538 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.094291 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f25d8-bcd5-41a9-8840-9131b8154538 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.094777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f25d8-bcd5-41a9-8840-9131b8154538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.119812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ef2f25d8-bcd5-41a9-8840-9131b8154538 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:33.121280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef2f25d8-bcd5-41a9-8840-9131b8154538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:33.128382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12370a7753c14f6082fadd2fb59bc65a', 'request_id': 'req-53727484-c872-47f7-88e9-98cabb5d36b6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.135041 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12370a7753c14f6082fadd2fb59bc65a', 'request_id': 'req-53727484-c872-47f7-88e9-98cabb5d36b6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.146616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12370a7753c14f6082fadd2fb59bc65a', 'request_id': 'req-53727484-c872-47f7-88e9-98cabb5d36b6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.162372 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] The token cache is being invalidate because project 12370a7753c14f6082fadd2fb59bc65a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:33.162650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.163308 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-53727484-c872-47f7-88e9-98cabb5d36b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.167973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1309/2632] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:33 2022] DELETE /identity/v3/projects/12370a7753c14f6082fadd2fb59bc65a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:33.169468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e605f-98c7-4e6c-ac64-53914347ec1e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.169764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e605f-98c7-4e6c-ac64-53914347ec1e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.170076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e605f-98c7-4e6c-ac64-53914347ec1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.192870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1324/2633] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:33.194257 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f25d8-bcd5-41a9-8840-9131b8154538 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:33.196979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef2f25d8-bcd5-41a9-8840-9131b8154538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:33.200027 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4e6e605f-98c7-4e6c-ac64-53914347ec1e None None] MFA Rules not processed for user `6d37f54f6721419b8428487153d9a89c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:33.201628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4e6e605f-98c7-4e6c-ac64-53914347ec1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:33.206234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:33.210917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f25d8-bcd5-41a9-8840-9131b8154538 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:33.217824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:33.218854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.219191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.219502 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] PATH_INFO: `/v3/users/a7a842043fcf4bcd9cbafaf43742d782` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.220687 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a842043fcf4bcd9cbafaf43742d782)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:33.222700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:33.273216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a842043fcf4bcd9cbafaf43742d782'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.285080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1310/2634] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:33.286217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e605f-98c7-4e6c-ac64-53914347ec1e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:33.290934 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a842043fcf4bcd9cbafaf43742d782'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.299408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a842043fcf4bcd9cbafaf43742d782'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.299600 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e605f-98c7-4e6c-ac64-53914347ec1e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:33.300340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a842043fcf4bcd9cbafaf43742d782'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.300968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a842043fcf4bcd9cbafaf43742d782'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.305468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd9f77-97b2-4422-aae9-5c05a85ff1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:33.306179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9f77-97b2-4422-aae9-5c05a85ff1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.306574 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9f77-97b2-4422-aae9-5c05a85ff1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.306749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9f77-97b2-4422-aae9-5c05a85ff1c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.307473 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd9f77-97b2-4422-aae9-5c05a85ff1c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:33.307780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd9f77-97b2-4422-aae9-5c05a85ff1c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:33.309393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd9f77-97b2-4422-aae9-5c05a85ff1c5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:33.318161 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1325/2635] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:33 2022] DELETE /identity/v3/users/a7a842043fcf4bcd9cbafaf43742d782 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:33.318932 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:33.320296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:33.326756 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1311/2636] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:33.331726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd9f77-97b2-4422-aae9-5c05a85ff1c5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:33.333253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-48cd9f77-97b2-4422-aae9-5c05a85ff1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:33.351042 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd9f77-97b2-4422-aae9-5c05a85ff1c5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:33.357085 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:33.456460 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0ccf2-37a9-490d-b84f-e012cb0eb609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:33.457898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0ccf2-37a9-490d-b84f-e012cb0eb609 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.458220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0ccf2-37a9-490d-b84f-e012cb0eb609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.458506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0ccf2-37a9-490d-b84f-e012cb0eb609 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.459705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0ccf2-37a9-490d-b84f-e012cb0eb609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:33.462346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0ccf2-37a9-490d-b84f-e012cb0eb609 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:33.464951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641c20f-8cc6-4a11-9df5-b599561bae07 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:33.476867 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa07766-0885-4de8-a4f8-1e5effea4938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:33.478131 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa07766-0885-4de8-a4f8-1e5effea4938 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.479558 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa07766-0885-4de8-a4f8-1e5effea4938 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.480266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa07766-0885-4de8-a4f8-1e5effea4938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.482904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8fa07766-0885-4de8-a4f8-1e5effea4938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:33.510190 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1312/2637] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:33 2022] POST /identity/v3/projects => generated 399 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:38:33.515415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0ccf2-37a9-490d-b84f-e012cb0eb609 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:33.571799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:33.591248 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa07766-0885-4de8-a4f8-1e5effea4938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:33.594615 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa07766-0885-4de8-a4f8-1e5effea4938 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:33.606175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1326/2638] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:33.620367 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0ccf2-37a9-490d-b84f-e012cb0eb609 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:33.626184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35869b2d-e969-4113-b33a-f8e782b7d9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:33.626889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35869b2d-e969-4113-b33a-f8e782b7d9cb admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.627173 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35869b2d-e969-4113-b33a-f8e782b7d9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.627454 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35869b2d-e969-4113-b33a-f8e782b7d9cb admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.628399 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35869b2d-e969-4113-b33a-f8e782b7d9cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:33.630423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35869b2d-e969-4113-b33a-f8e782b7d9cb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:33.659463 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa07766-0885-4de8-a4f8-1e5effea4938 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:33.666571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8fa07766-0885-4de8-a4f8-1e5effea4938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:33.680066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1313/2639] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:33 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:33.683414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35869b2d-e969-4113-b33a-f8e782b7d9cb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:33.702754 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa07766-0885-4de8-a4f8-1e5effea4938 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:33.709864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35869b2d-e969-4113-b33a-f8e782b7d9cb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:33.714710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:33.714710 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.722471 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.726251 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] PATH_INFO: `/v3/projects/8f0aa35b2d4f416688d88754a5be87f9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.726419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342cc2c-0c79-45fb-a83e-5b6547ed34de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:33.727009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342cc2c-0c79-45fb-a83e-5b6547ed34de admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.727307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342cc2c-0c79-45fb-a83e-5b6547ed34de admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.727637 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342cc2c-0c79-45fb-a83e-5b6547ed34de admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.728401 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342cc2c-0c79-45fb-a83e-5b6547ed34de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:33.728898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f0aa35b2d4f416688d88754a5be87f9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:33.730159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342cc2c-0c79-45fb-a83e-5b6547ed34de admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:33.731910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:33.756601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1314/2640] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:33.757392 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342cc2c-0c79-45fb-a83e-5b6547ed34de admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:33.777363 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342cc2c-0c79-45fb-a83e-5b6547ed34de admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:33.787281 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7912b-3897-4dae-8949-c678d1f1c706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:33.787483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7912b-3897-4dae-8949-c678d1f1c706 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.787733 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7912b-3897-4dae-8949-c678d1f1c706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.788016 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7912b-3897-4dae-8949-c678d1f1c706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.788776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7912b-3897-4dae-8949-c678d1f1c706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:33.792930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7912b-3897-4dae-8949-c678d1f1c706 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:33.805542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1315/2641] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:33.809531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f0aa35b2d4f416688d88754a5be87f9', 'request_id': 'req-ce921ae3-2524-409d-bb23-360a26414acb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.814983 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7912b-3897-4dae-8949-c678d1f1c706 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:33.818929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f0aa35b2d4f416688d88754a5be87f9', 'request_id': 'req-ce921ae3-2524-409d-bb23-360a26414acb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.827836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f0aa35b2d4f416688d88754a5be87f9', 'request_id': 'req-ce921ae3-2524-409d-bb23-360a26414acb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.828682 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7912b-3897-4dae-8949-c678d1f1c706 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:33.842795 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] The token cache is being invalidate because project 8f0aa35b2d4f416688d88754a5be87f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:33.843049 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.844246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ce921ae3-2524-409d-bb23-360a26414acb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:33.847747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1327/2642] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:33 2022] DELETE /identity/v3/projects/8f0aa35b2d4f416688d88754a5be87f9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:33.853129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c46f0-1040-418f-9c9a-93f03bbf00a4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.854203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c46f0-1040-418f-9c9a-93f03bbf00a4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.854629 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c46f0-1040-418f-9c9a-93f03bbf00a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.888129 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-082c46f0-1040-418f-9c9a-93f03bbf00a4 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:33.889335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-082c46f0-1040-418f-9c9a-93f03bbf00a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:33.913433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d1a21-b14c-4ee9-bc00-4c75f4d04d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:33.914797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1a21-b14c-4ee9-bc00-4c75f4d04d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.915576 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1a21-b14c-4ee9-bc00-4c75f4d04d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.916200 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1a21-b14c-4ee9-bc00-4c75f4d04d9e admin admin] PATH_INFO: `/v3/projects/f0dee734971f432e93f1d860790ab325/users/687e102b71a342c2ade6050106540cc9/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:33.928094 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:33.945219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d1a21-b14c-4ee9-bc00-4c75f4d04d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0dee734971f432e93f1d860790ab325, user_id=687e102b71a342c2ade6050106540cc9, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:33.948692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d1a21-b14c-4ee9-bc00-4c75f4d04d9e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:33.963774 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1316/2643] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:33 2022] PUT /identity/v3/projects/f0dee734971f432e93f1d860790ab325/users/687e102b71a342c2ade6050106540cc9/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:33.970226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d1a21-b14c-4ee9-bc00-4c75f4d04d9e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:33.970226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3d2d1a21-b14c-4ee9-bc00-4c75f4d04d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:33.981347 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1328/2644] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:33.989981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df217717-ad5e-4c57-a43c-3372d67f12b5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:33.989981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df217717-ad5e-4c57-a43c-3372d67f12b5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:33.989981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df217717-ad5e-4c57-a43c-3372d67f12b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.019273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-df217717-ad5e-4c57-a43c-3372d67f12b5 None None] MFA Rules not processed for user `687e102b71a342c2ade6050106540cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:34.020482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-df217717-ad5e-4c57-a43c-3372d67f12b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:34.050805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d1a21-b14c-4ee9-bc00-4c75f4d04d9e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:34.058662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:34.060456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.061237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.061237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] PATH_INFO: `/v3/users/6d37f54f6721419b8428487153d9a89c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.063534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d37f54f6721419b8428487153d9a89c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:34.065502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:34.097553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:34.105810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1329/2645] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:34.105810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa591741-1998-4d6b-a255-5e7eb6145744 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.105810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa591741-1998-4d6b-a255-5e7eb6145744 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.105810 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa591741-1998-4d6b-a255-5e7eb6145744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.125885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d37f54f6721419b8428487153d9a89c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:34.127711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d37f54f6721419b8428487153d9a89c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:34.133326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fa591741-1998-4d6b-a255-5e7eb6145744 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:34.134575 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fa591741-1998-4d6b-a255-5e7eb6145744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:34.139484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d37f54f6721419b8428487153d9a89c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:34.144809 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1330/2646] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:34.145657 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa591741-1998-4d6b-a255-5e7eb6145744 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:34.147039 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fa591741-1998-4d6b-a255-5e7eb6145744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:34.154999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d37f54f6721419b8428487153d9a89c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:34.157608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d37f54f6721419b8428487153d9a89c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:34.171536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1317/2647] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:33 2022] DELETE /identity/v3/users/6d37f54f6721419b8428487153d9a89c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:34.173165 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:34.214473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa591741-1998-4d6b-a255-5e7eb6145744 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:34.244075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cbde2-8ae0-4ff0-b6f7-290bbdd849b0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:34.251498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197bcba3-d816-4dd7-82bd-d5572bcfea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:34.252334 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bcba3-d816-4dd7-82bd-d5572bcfea07 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.252627 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bcba3-d816-4dd7-82bd-d5572bcfea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.252901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bcba3-d816-4dd7-82bd-d5572bcfea07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.254538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197bcba3-d816-4dd7-82bd-d5572bcfea07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:34.254861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197bcba3-d816-4dd7-82bd-d5572bcfea07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:34.256676 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197bcba3-d816-4dd7-82bd-d5572bcfea07 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:34.259850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:34.271634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e3ec8-6c0c-49dd-b8af-e48a2da202cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:34.272300 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e3ec8-6c0c-49dd-b8af-e48a2da202cc service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.272582 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e3ec8-6c0c-49dd-b8af-e48a2da202cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.272853 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e3ec8-6c0c-49dd-b8af-e48a2da202cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.274203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1318/2648] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:34.274808 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-699e3ec8-6c0c-49dd-b8af-e48a2da202cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:34.276958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054b19c-72c9-4ddd-a636-8d53a65e1d66 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.278346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054b19c-72c9-4ddd-a636-8d53a65e1d66 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.278806 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054b19c-72c9-4ddd-a636-8d53a65e1d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.300723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a054b19c-72c9-4ddd-a636-8d53a65e1d66 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:34.302044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a054b19c-72c9-4ddd-a636-8d53a65e1d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:34.314221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1319/2649] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:34.315130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054b19c-72c9-4ddd-a636-8d53a65e1d66 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:34.326010 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054b19c-72c9-4ddd-a636-8d53a65e1d66 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:34.331297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85854c-9ac2-4c5c-8f61-dac7adf8e867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:34.331768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85854c-9ac2-4c5c-8f61-dac7adf8e867 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.332051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85854c-9ac2-4c5c-8f61-dac7adf8e867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.332332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85854c-9ac2-4c5c-8f61-dac7adf8e867 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.333031 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85854c-9ac2-4c5c-8f61-dac7adf8e867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:34.334989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85854c-9ac2-4c5c-8f61-dac7adf8e867 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:34.354896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e3ec8-6c0c-49dd-b8af-e48a2da202cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:34.356369 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e3ec8-6c0c-49dd-b8af-e48a2da202cc service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:34.369206 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1331/2650] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:34.369747 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1320/2651] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:34 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:34.372100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e3ec8-6c0c-49dd-b8af-e48a2da202cc service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:34.375475 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85854c-9ac2-4c5c-8f61-dac7adf8e867 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:34.384211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-699e3ec8-6c0c-49dd-b8af-e48a2da202cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:34.467906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85854c-9ac2-4c5c-8f61-dac7adf8e867 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:34.480882 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcfc1a-0db7-42b1-b738-ceadd2680461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:34.481584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcfc1a-0db7-42b1-b738-ceadd2680461 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.481871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcfc1a-0db7-42b1-b738-ceadd2680461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.482179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcfc1a-0db7-42b1-b738-ceadd2680461 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.482925 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcfc1a-0db7-42b1-b738-ceadd2680461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:34.484969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcfc1a-0db7-42b1-b738-ceadd2680461 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:34.493069 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e3ec8-6c0c-49dd-b8af-e48a2da202cc service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:34.520627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:34.542282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1321/2652] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:34 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:34.550271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcfc1a-0db7-42b1-b738-ceadd2680461 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:34.565439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcfc1a-0db7-42b1-b738-ceadd2680461 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:34.571904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92c4d1-9c7f-4b61-9e72-e8737aaeb653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:34.572307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5ac62-a5df-4f7b-a4e5-3d226f8b1eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:34.572583 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92c4d1-9c7f-4b61-9e72-e8737aaeb653 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.572868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92c4d1-9c7f-4b61-9e72-e8737aaeb653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.573147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92c4d1-9c7f-4b61-9e72-e8737aaeb653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.573469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5ac62-a5df-4f7b-a4e5-3d226f8b1eaa service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.573862 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5ac62-a5df-4f7b-a4e5-3d226f8b1eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.574087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92c4d1-9c7f-4b61-9e72-e8737aaeb653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:34.574457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5ac62-a5df-4f7b-a4e5-3d226f8b1eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.575591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92c4d1-9c7f-4b61-9e72-e8737aaeb653 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:34.591616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1322/2653] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:34.593353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92c4d1-9c7f-4b61-9e72-e8737aaeb653 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:34.600820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5ac62-a5df-4f7b-a4e5-3d226f8b1eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:34.602395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5ac62-a5df-4f7b-a4e5-3d226f8b1eaa service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:34.607356 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92c4d1-9c7f-4b61-9e72-e8737aaeb653 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:34.615177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832526ac-d7a5-47a0-a18e-c6ed2e5de339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:34.615942 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832526ac-d7a5-47a0-a18e-c6ed2e5de339 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.616299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832526ac-d7a5-47a0-a18e-c6ed2e5de339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.616696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832526ac-d7a5-47a0-a18e-c6ed2e5de339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.617569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832526ac-d7a5-47a0-a18e-c6ed2e5de339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:34.619576 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832526ac-d7a5-47a0-a18e-c6ed2e5de339 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:34.628714 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1332/2654] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:34.637153 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1323/2655] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:38:34.638483 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5ac62-a5df-4f7b-a4e5-3d226f8b1eaa service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:34.648993 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5ac62-a5df-4f7b-a4e5-3d226f8b1eaa service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:34.654547 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21107fad-bc88-418a-bd80-b827cc6b886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:34.655453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21107fad-bc88-418a-bd80-b827cc6b886f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.655738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21107fad-bc88-418a-bd80-b827cc6b886f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.656052 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21107fad-bc88-418a-bd80-b827cc6b886f admin admin] PATH_INFO: `/v3/projects/841812e5e5b6442f847eef112c8ed38c/users/d89fb64a96db407ba451bc54fca2c8ab/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.680352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21107fad-bc88-418a-bd80-b827cc6b886f admin admin] RBAC: Authorizing `identity:create_grant(project_id=841812e5e5b6442f847eef112c8ed38c, user_id=d89fb64a96db407ba451bc54fca2c8ab, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:34.684266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21107fad-bc88-418a-bd80-b827cc6b886f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:34.708407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1333/2656] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:34 2022] PUT /identity/v3/projects/841812e5e5b6442f847eef112c8ed38c/users/d89fb64a96db407ba451bc54fca2c8ab/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:34.718663 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9cc85-522e-490d-b675-e583b54889f4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.719057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9cc85-522e-490d-b675-e583b54889f4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.719905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9cc85-522e-490d-b675-e583b54889f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.771990 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1bc9cc85-522e-490d-b675-e583b54889f4 None None] MFA Rules not processed for user `d89fb64a96db407ba451bc54fca2c8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:34.773687 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1bc9cc85-522e-490d-b675-e583b54889f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:34.861536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1324/2657] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:34.866256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21107fad-bc88-418a-bd80-b827cc6b886f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:34.986053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21107fad-bc88-418a-bd80-b827cc6b886f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:34.992556 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30757d-5d80-468b-ad29-5520e4b0beb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:34.998196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30757d-5d80-468b-ad29-5520e4b0beb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:34.998196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30757d-5d80-468b-ad29-5520e4b0beb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:34.998196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30757d-5d80-468b-ad29-5520e4b0beb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:34.998196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30757d-5d80-468b-ad29-5520e4b0beb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:34.998196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30757d-5d80-468b-ad29-5520e4b0beb6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.081612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1334/2658] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:34 2022] POST /identity/v3/projects => generated 379 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:35.082829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9cc85-522e-490d-b675-e583b54889f4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.097978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9cc85-522e-490d-b675-e583b54889f4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.134359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:35.185932 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95b57d-8d78-42f2-8d2e-adaff0584a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.186648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95b57d-8d78-42f2-8d2e-adaff0584a11 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.186936 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95b57d-8d78-42f2-8d2e-adaff0584a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.187217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95b57d-8d78-42f2-8d2e-adaff0584a11 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.187958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95b57d-8d78-42f2-8d2e-adaff0584a11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.189906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95b57d-8d78-42f2-8d2e-adaff0584a11 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.210926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30757d-5d80-468b-ad29-5520e4b0beb6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.214672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7e30757d-5d80-468b-ad29-5520e4b0beb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:35.216840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1325/2659] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:35 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:38:35.222830 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95b57d-8d78-42f2-8d2e-adaff0584a11 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.238037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95b57d-8d78-42f2-8d2e-adaff0584a11 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.242317 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30757d-5d80-468b-ad29-5520e4b0beb6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.244680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913a0c9-8e64-4715-9470-69d5fb32663c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.245353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913a0c9-8e64-4715-9470-69d5fb32663c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.245644 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913a0c9-8e64-4715-9470-69d5fb32663c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.245951 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913a0c9-8e64-4715-9470-69d5fb32663c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.246689 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913a0c9-8e64-4715-9470-69d5fb32663c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.249066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913a0c9-8e64-4715-9470-69d5fb32663c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.251630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.252607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.252964 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.253254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] PATH_INFO: `/v3/projects/e707a56bc1944837b26f6306784d3012` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.262868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e707a56bc1944837b26f6306784d3012)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.263373 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1326/2660] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:35.266889 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913a0c9-8e64-4715-9470-69d5fb32663c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.268095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.277580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913a0c9-8e64-4715-9470-69d5fb32663c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.284623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7b764-e63b-444e-a298-08e8aedb26cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.285694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b764-e63b-444e-a298-08e8aedb26cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.285998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b764-e63b-444e-a298-08e8aedb26cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.286282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b764-e63b-444e-a298-08e8aedb26cc admin admin] PATH_INFO: `/v3/projects/bfa8df4cda7e4dcbb43742d3b9c82561/users/79a0667c7e2e4211bba3f2807567a531/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.307295 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7b764-e63b-444e-a298-08e8aedb26cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa8df4cda7e4dcbb43742d3b9c82561, user_id=79a0667c7e2e4211bba3f2807567a531, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.310222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7b764-e63b-444e-a298-08e8aedb26cc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.328608 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1327/2661] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:35 2022] PUT /identity/v3/projects/bfa8df4cda7e4dcbb43742d3b9c82561/users/79a0667c7e2e4211bba3f2807567a531/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:35.332293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7b764-e63b-444e-a298-08e8aedb26cc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.347528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e707a56bc1944837b26f6306784d3012', 'request_id': 'req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:35.360113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e707a56bc1944837b26f6306784d3012', 'request_id': 'req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:35.368830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e707a56bc1944837b26f6306784d3012', 'request_id': 'req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:35.381905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] The token cache is being invalidate because project e707a56bc1944837b26f6306784d3012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:35.382290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:35.382927 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d445bdc8-cb16-4cad-8a1c-9c955fe55c8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:35.388005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1335/2662] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:35 2022] DELETE /identity/v3/projects/e707a56bc1944837b26f6306784d3012 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:35.389785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d8675-130a-477a-86c7-866029156382 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.390116 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d8675-130a-477a-86c7-866029156382 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.390402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d8675-130a-477a-86c7-866029156382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.409804 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:35.417282 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-735d8675-130a-477a-86c7-866029156382 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:35.418507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-735d8675-130a-477a-86c7-866029156382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:35.421222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7b764-e63b-444e-a298-08e8aedb26cc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.492796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0a8ba-8315-43ee-b514-ddfae50a22c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.493479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0a8ba-8315-43ee-b514-ddfae50a22c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.493768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0a8ba-8315-43ee-b514-ddfae50a22c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.494086 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0a8ba-8315-43ee-b514-ddfae50a22c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.494825 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0a8ba-8315-43ee-b514-ddfae50a22c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.495734 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1336/2663] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:35.496943 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0a8ba-8315-43ee-b514-ddfae50a22c1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.504093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c89aa-3c1a-470a-9387-b4dcb9eca46c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.504386 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c89aa-3c1a-470a-9387-b4dcb9eca46c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.504658 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c89aa-3c1a-470a-9387-b4dcb9eca46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.509799 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1328/2664] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:38:35.513565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0a8ba-8315-43ee-b514-ddfae50a22c1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.515537 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ffc0a8ba-8315-43ee-b514-ddfae50a22c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:35.529045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0a8ba-8315-43ee-b514-ddfae50a22c1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.530253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cf1c89aa-3c1a-470a-9387-b4dcb9eca46c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:35.531551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cf1c89aa-3c1a-470a-9387-b4dcb9eca46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:35.534417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33190bc-b4f7-40cd-a3f9-fde6030526f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.535314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33190bc-b4f7-40cd-a3f9-fde6030526f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.535602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33190bc-b4f7-40cd-a3f9-fde6030526f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.535884 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33190bc-b4f7-40cd-a3f9-fde6030526f1 admin admin] PATH_INFO: `/v3/projects/bfa8df4cda7e4dcbb43742d3b9c82561/users/79a0667c7e2e4211bba3f2807567a531/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.538108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33190bc-b4f7-40cd-a3f9-fde6030526f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa8df4cda7e4dcbb43742d3b9c82561, user_id=79a0667c7e2e4211bba3f2807567a531, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.540750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33190bc-b4f7-40cd-a3f9-fde6030526f1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.540883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1337/2665] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:35.544933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c89aa-3c1a-470a-9387-b4dcb9eca46c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.553381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1329/2666] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:35 2022] PUT /identity/v3/projects/bfa8df4cda7e4dcbb43742d3b9c82561/users/79a0667c7e2e4211bba3f2807567a531/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:35.556090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c89aa-3c1a-470a-9387-b4dcb9eca46c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.565237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58531b96-e5e7-4ac0-bd49-e2dd5d65a34e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.565536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58531b96-e5e7-4ac0-bd49-e2dd5d65a34e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.565824 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58531b96-e5e7-4ac0-bd49-e2dd5d65a34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.585346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:35.598587 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-58531b96-e5e7-4ac0-bd49-e2dd5d65a34e None None] MFA Rules not processed for user `79a0667c7e2e4211bba3f2807567a531`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:35.601798 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-58531b96-e5e7-4ac0-bd49-e2dd5d65a34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:35.617415 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9708c00-c451-4174-9ba1-8c1df77161b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.618153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9708c00-c451-4174-9ba1-8c1df77161b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.618440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9708c00-c451-4174-9ba1-8c1df77161b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.618849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9708c00-c451-4174-9ba1-8c1df77161b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.619598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9708c00-c451-4174-9ba1-8c1df77161b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:35.619907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9708c00-c451-4174-9ba1-8c1df77161b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.621703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9708c00-c451-4174-9ba1-8c1df77161b7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.635172 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1338/2667] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:38:35.639383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9708c00-c451-4174-9ba1-8c1df77161b7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.652030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9708c00-c451-4174-9ba1-8c1df77161b7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.658159 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd663f-8386-4e4b-b0e4-a5d97510cf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.658904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd663f-8386-4e4b-b0e4-a5d97510cf98 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.659339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd663f-8386-4e4b-b0e4-a5d97510cf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.659625 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd663f-8386-4e4b-b0e4-a5d97510cf98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.660340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd663f-8386-4e4b-b0e4-a5d97510cf98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.662295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd663f-8386-4e4b-b0e4-a5d97510cf98 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.679512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1330/2668] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:35.693183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58531b96-e5e7-4ac0-bd49-e2dd5d65a34e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.694771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-58531b96-e5e7-4ac0-bd49-e2dd5d65a34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:35.701684 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1339/2669] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:35 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:38:35.705747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd663f-8386-4e4b-b0e4-a5d97510cf98 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.762186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58531b96-e5e7-4ac0-bd49-e2dd5d65a34e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.784232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd663f-8386-4e4b-b0e4-a5d97510cf98 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.789170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee875a-9cf2-4c3a-bfef-2ec5a042c8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.789901 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee875a-9cf2-4c3a-bfef-2ec5a042c8c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.790433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee875a-9cf2-4c3a-bfef-2ec5a042c8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.790723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee875a-9cf2-4c3a-bfef-2ec5a042c8c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.791446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee875a-9cf2-4c3a-bfef-2ec5a042c8c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.793397 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee875a-9cf2-4c3a-bfef-2ec5a042c8c2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.817013 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1340/2670] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:35 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:38:35.821166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee875a-9cf2-4c3a-bfef-2ec5a042c8c2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.824805 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859018e-c88c-4933-baa3-69e3c6649bcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.826108 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859018e-c88c-4933-baa3-69e3c6649bcb service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.826598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:35.826762 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859018e-c88c-4933-baa3-69e3c6649bcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.827014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859018e-c88c-4933-baa3-69e3c6649bcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.833434 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee875a-9cf2-4c3a-bfef-2ec5a042c8c2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.841245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7c895-2679-4968-8bd4-1c008b3285b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.842581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7c895-2679-4968-8bd4-1c008b3285b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.842887 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7c895-2679-4968-8bd4-1c008b3285b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.843177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7c895-2679-4968-8bd4-1c008b3285b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.843965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7c895-2679-4968-8bd4-1c008b3285b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.845599 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7c895-2679-4968-8bd4-1c008b3285b6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.858246 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1341/2671] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:38:35.862731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7c895-2679-4968-8bd4-1c008b3285b6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.872985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7c895-2679-4968-8bd4-1c008b3285b6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.878392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58ce4b-b47d-41e3-ae13-41336f994e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.879076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58ce4b-b47d-41e3-ae13-41336f994e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.879362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58ce4b-b47d-41e3-ae13-41336f994e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.879650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58ce4b-b47d-41e3-ae13-41336f994e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.880395 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58ce4b-b47d-41e3-ae13-41336f994e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.882025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58ce4b-b47d-41e3-ae13-41336f994e4a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.893833 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1342/2672] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:38:35.897986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58ce4b-b47d-41e3-ae13-41336f994e4a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:35.908956 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58ce4b-b47d-41e3-ae13-41336f994e4a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:35.910912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859018e-c88c-4933-baa3-69e3c6649bcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.912803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859018e-c88c-4933-baa3-69e3c6649bcb service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.915199 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083118a4-12e4-4ede-958c-3b3b78a881e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:35.916451 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083118a4-12e4-4ede-958c-3b3b78a881e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.916774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083118a4-12e4-4ede-958c-3b3b78a881e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.917066 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083118a4-12e4-4ede-958c-3b3b78a881e1 admin admin] PATH_INFO: `/v3/projects/7738548b6bed49abadd5556f90281920/users/bd7bb79340404274a85e71bb8fb5997e/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:35.919782 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:35.928705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1331/2673] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:38:35.938800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083118a4-12e4-4ede-958c-3b3b78a881e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7738548b6bed49abadd5556f90281920, user_id=bd7bb79340404274a85e71bb8fb5997e, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:35.941717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083118a4-12e4-4ede-958c-3b3b78a881e1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:35.961994 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1343/2674] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:35 2022] PUT /identity/v3/projects/7738548b6bed49abadd5556f90281920/users/bd7bb79340404274a85e71bb8fb5997e/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:35.967855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270a233-1f23-441c-9b12-21406eb87b33 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:35.968057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270a233-1f23-441c-9b12-21406eb87b33 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:35.968336 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270a233-1f23-441c-9b12-21406eb87b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.007232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c270a233-1f23-441c-9b12-21406eb87b33 None None] MFA Rules not processed for user `bd7bb79340404274a85e71bb8fb5997e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:36.008468 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c270a233-1f23-441c-9b12-21406eb87b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:36.085973 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1332/2675] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:36.088235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083118a4-12e4-4ede-958c-3b3b78a881e1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:36.180405 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083118a4-12e4-4ede-958c-3b3b78a881e1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:36.185254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd6690-940c-4f36-a1e0-47cd506c81e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:36.186146 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd6690-940c-4f36-a1e0-47cd506c81e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.186500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd6690-940c-4f36-a1e0-47cd506c81e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.187090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd6690-940c-4f36-a1e0-47cd506c81e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.187509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd6690-940c-4f36-a1e0-47cd506c81e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:36.189666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd6690-940c-4f36-a1e0-47cd506c81e5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:36.222491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1344/2676] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:36 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:36.227087 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270a233-1f23-441c-9b12-21406eb87b33 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:36.241187 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270a233-1f23-441c-9b12-21406eb87b33 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:36.320932 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39599f-2016-4357-ba6d-6f8cb6d1d60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:36.321692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39599f-2016-4357-ba6d-6f8cb6d1d60d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.322013 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39599f-2016-4357-ba6d-6f8cb6d1d60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.322346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39599f-2016-4357-ba6d-6f8cb6d1d60d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.323074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39599f-2016-4357-ba6d-6f8cb6d1d60d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:36.325411 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39599f-2016-4357-ba6d-6f8cb6d1d60d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:36.353989 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1333/2677] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:36 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:38:36.357562 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd6690-940c-4f36-a1e0-47cd506c81e5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:36.369839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd6690-940c-4f36-a1e0-47cd506c81e5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:36.377424 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65dc62-192e-4918-8b22-df80dce84ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:36.378175 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65dc62-192e-4918-8b22-df80dce84ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.378598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65dc62-192e-4918-8b22-df80dce84ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.378918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65dc62-192e-4918-8b22-df80dce84ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.379759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65dc62-192e-4918-8b22-df80dce84ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:36.381591 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65dc62-192e-4918-8b22-df80dce84ab9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:36.394097 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1345/2678] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:38:36.398480 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39599f-2016-4357-ba6d-6f8cb6d1d60d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:36.410270 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39599f-2016-4357-ba6d-6f8cb6d1d60d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:36.416303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e67eac-dfac-4cd2-b83f-5a843e25ace8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:36.417349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e67eac-dfac-4cd2-b83f-5a843e25ace8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.417634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e67eac-dfac-4cd2-b83f-5a843e25ace8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.417912 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e67eac-dfac-4cd2-b83f-5a843e25ace8 admin admin] PATH_INFO: `/v3/projects/a13e784219d948f6a7d6be6df8af6c97/users/fc607670b44e4d9197655743d6285adf/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.434870 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e67eac-dfac-4cd2-b83f-5a843e25ace8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a13e784219d948f6a7d6be6df8af6c97, user_id=fc607670b44e4d9197655743d6285adf, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:36.437791 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e67eac-dfac-4cd2-b83f-5a843e25ace8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:36.451777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1334/2679] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:36 2022] PUT /identity/v3/projects/a13e784219d948f6a7d6be6df8af6c97/users/fc607670b44e4d9197655743d6285adf/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:36.456721 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65dc62-192e-4918-8b22-df80dce84ab9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:36.467717 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65dc62-192e-4918-8b22-df80dce84ab9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:36.530045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e68c2-a1f2-4c70-b3b2-6337fc8bceef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:36.530716 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e68c2-a1f2-4c70-b3b2-6337fc8bceef admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.531122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e68c2-a1f2-4c70-b3b2-6337fc8bceef admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.531413 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e68c2-a1f2-4c70-b3b2-6337fc8bceef admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.532614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e68c2-a1f2-4c70-b3b2-6337fc8bceef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:36.534226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e68c2-a1f2-4c70-b3b2-6337fc8bceef admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:36.535006 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e67eac-dfac-4cd2-b83f-5a843e25ace8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:36.547851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1346/2680] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:36.548048 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e67eac-dfac-4cd2-b83f-5a843e25ace8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:36.548889 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e68c2-a1f2-4c70-b3b2-6337fc8bceef admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:36.551969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdb022-6286-4920-833e-9778318a6ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:36.552841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdb022-6286-4920-833e-9778318a6ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.553129 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdb022-6286-4920-833e-9778318a6ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.553415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdb022-6286-4920-833e-9778318a6ee7 admin admin] PATH_INFO: `/v3/users/d89fb64a96db407ba451bc54fca2c8ab` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.554656 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdb022-6286-4920-833e-9778318a6ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89fb64a96db407ba451bc54fca2c8ab)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:36.556704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdb022-6286-4920-833e-9778318a6ee7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:36.558707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e68c2-a1f2-4c70-b3b2-6337fc8bceef admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:36.564353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47bcf9-f510-43d5-9e6d-6ad6df306347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:36.565249 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47bcf9-f510-43d5-9e6d-6ad6df306347 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.565531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47bcf9-f510-43d5-9e6d-6ad6df306347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.565832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47bcf9-f510-43d5-9e6d-6ad6df306347 admin admin] PATH_INFO: `/v3/projects/a13e784219d948f6a7d6be6df8af6c97/users/fc607670b44e4d9197655743d6285adf/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.568155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47bcf9-f510-43d5-9e6d-6ad6df306347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a13e784219d948f6a7d6be6df8af6c97, user_id=fc607670b44e4d9197655743d6285adf, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:36.570790 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47bcf9-f510-43d5-9e6d-6ad6df306347 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:36.582041 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1347/2681] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:36 2022] PUT /identity/v3/projects/a13e784219d948f6a7d6be6df8af6c97/users/fc607670b44e4d9197655743d6285adf/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:36.593784 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9306154-d506-4b7d-a4b7-d8ca21b7ddb6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.594399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9306154-d506-4b7d-a4b7-d8ca21b7ddb6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.594805 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9306154-d506-4b7d-a4b7-d8ca21b7ddb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.609964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8bdb022-6286-4920-833e-9778318a6ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89fb64a96db407ba451bc54fca2c8ab'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:36.610664 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8bdb022-6286-4920-833e-9778318a6ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89fb64a96db407ba451bc54fca2c8ab'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:36.612976 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:36.620276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:36.620865 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8bdb022-6286-4920-833e-9778318a6ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89fb64a96db407ba451bc54fca2c8ab'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:36.623273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a9306154-d506-4b7d-a4b7-d8ca21b7ddb6 None None] MFA Rules not processed for user `fc607670b44e4d9197655743d6285adf`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:36.625967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a9306154-d506-4b7d-a4b7-d8ca21b7ddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:36.635556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8bdb022-6286-4920-833e-9778318a6ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89fb64a96db407ba451bc54fca2c8ab'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:36.636210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8bdb022-6286-4920-833e-9778318a6ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89fb64a96db407ba451bc54fca2c8ab'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:36.650918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1335/2682] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:36 2022] DELETE /identity/v3/users/d89fb64a96db407ba451bc54fca2c8ab => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:36.655027 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d4fcf-a711-442e-acc3-c835aef09961 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.655328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d4fcf-a711-442e-acc3-c835aef09961 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.655604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d4fcf-a711-442e-acc3-c835aef09961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.672953 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cc4d4fcf-a711-442e-acc3-c835aef09961 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:36.674182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cc4d4fcf-a711-442e-acc3-c835aef09961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:36.697917 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1348/2683] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:36 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:36.710261 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9306154-d506-4b7d-a4b7-d8ca21b7ddb6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:36.711608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a9306154-d506-4b7d-a4b7-d8ca21b7ddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:36.736326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1336/2684] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:36.746489 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d4fcf-a711-442e-acc3-c835aef09961 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:36.747554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cc4d4fcf-a711-442e-acc3-c835aef09961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:36.772100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9306154-d506-4b7d-a4b7-d8ca21b7ddb6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:36.806727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d4fcf-a711-442e-acc3-c835aef09961 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:36.810443 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf405a-96e6-41f8-9eca-c9f373e92dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:36.811161 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf405a-96e6-41f8-9eca-c9f373e92dff service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.811455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf405a-96e6-41f8-9eca-c9f373e92dff service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.811738 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf405a-96e6-41f8-9eca-c9f373e92dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.821457 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf405a-96e6-41f8-9eca-c9f373e92dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:36.822585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c794161-2857-4398-a621-7ca23740ad9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:36.822934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf405a-96e6-41f8-9eca-c9f373e92dff service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:36.823239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c794161-2857-4398-a621-7ca23740ad9a service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.823523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c794161-2857-4398-a621-7ca23740ad9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.823801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c794161-2857-4398-a621-7ca23740ad9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.833502 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1337/2685] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:36.887974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c794161-2857-4398-a621-7ca23740ad9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:36.889406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c794161-2857-4398-a621-7ca23740ad9a service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:36.899372 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1349/2686] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:36 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:38:36.974969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf405a-96e6-41f8-9eca-c9f373e92dff service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:36.987911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf405a-96e6-41f8-9eca-c9f373e92dff service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:36.991920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7dfcc-a5ff-4e6d-9bcf-d2e6de370b0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:36.992742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7dfcc-a5ff-4e6d-9bcf-d2e6de370b0c service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:36.993096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7dfcc-a5ff-4e6d-9bcf-d2e6de370b0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:36.993387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7dfcc-a5ff-4e6d-9bcf-d2e6de370b0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:36.995341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cdf7dfcc-a5ff-4e6d-9bcf-d2e6de370b0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:37.062817 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7dfcc-a5ff-4e6d-9bcf-d2e6de370b0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:37.064320 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7dfcc-a5ff-4e6d-9bcf-d2e6de370b0c service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:37.073981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1338/2687] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:37.115298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c794161-2857-4398-a621-7ca23740ad9a service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:37.116751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1c794161-2857-4398-a621-7ca23740ad9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:37.128902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:37.134738 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c794161-2857-4398-a621-7ca23740ad9a service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:37.141569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:37.142559 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:37.142849 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:37.143125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] PATH_INFO: `/v3/projects/841812e5e5b6442f847eef112c8ed38c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:37.144403 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=841812e5e5b6442f847eef112c8ed38c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:37.147295 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:37.206725 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841812e5e5b6442f847eef112c8ed38c', 'request_id': 'req-9ddb48a8-6b37-468b-bab9-a43e0de48e92'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.215231 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841812e5e5b6442f847eef112c8ed38c', 'request_id': 'req-9ddb48a8-6b37-468b-bab9-a43e0de48e92'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.224030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841812e5e5b6442f847eef112c8ed38c', 'request_id': 'req-9ddb48a8-6b37-468b-bab9-a43e0de48e92'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.235072 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] The token cache is being invalidate because project 841812e5e5b6442f847eef112c8ed38c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:37.235362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.236095 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.237755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1350/2688] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:37 2022] DELETE /identity/v3/projects/841812e5e5b6442f847eef112c8ed38c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:37.241667 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7dfcc-a5ff-4e6d-9bcf-d2e6de370b0c service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:37.252979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7dfcc-a5ff-4e6d-9bcf-d2e6de370b0c service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:37.259691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:37.314850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6eabf-66a4-489e-8301-4be0d1af173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:37.315726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6eabf-66a4-489e-8301-4be0d1af173a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:37.316012 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6eabf-66a4-489e-8301-4be0d1af173a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:37.316291 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6eabf-66a4-489e-8301-4be0d1af173a admin admin] PATH_INFO: `/v3/users/79a0667c7e2e4211bba3f2807567a531` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:37.317503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6eabf-66a4-489e-8301-4be0d1af173a admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a0667c7e2e4211bba3f2807567a531)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:37.319458 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6eabf-66a4-489e-8301-4be0d1af173a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:37.366274 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9fa6eabf-66a4-489e-8301-4be0d1af173a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a0667c7e2e4211bba3f2807567a531'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.366964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9fa6eabf-66a4-489e-8301-4be0d1af173a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a0667c7e2e4211bba3f2807567a531'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.374192 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9fa6eabf-66a4-489e-8301-4be0d1af173a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a0667c7e2e4211bba3f2807567a531'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.386590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9fa6eabf-66a4-489e-8301-4be0d1af173a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a0667c7e2e4211bba3f2807567a531'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.387270 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9fa6eabf-66a4-489e-8301-4be0d1af173a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a0667c7e2e4211bba3f2807567a531'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.399755 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1339/2689] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:37 2022] DELETE /identity/v3/users/79a0667c7e2e4211bba3f2807567a531 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:37.667012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:37.668542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:37.682982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb48a8-6b37-468b-bab9-a43e0de48e92 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:37.740182 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:37.741055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:37.741340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:37.741620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] PATH_INFO: `/v3/projects/bfa8df4cda7e4dcbb43742d3b9c82561` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:37.742834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa8df4cda7e4dcbb43742d3b9c82561)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:37.744774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:37.789741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa8df4cda7e4dcbb43742d3b9c82561', 'request_id': 'req-c3d29fd9-074d-431d-ac57-923e8bb43ce6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.797904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa8df4cda7e4dcbb43742d3b9c82561', 'request_id': 'req-c3d29fd9-074d-431d-ac57-923e8bb43ce6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.803834 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa8df4cda7e4dcbb43742d3b9c82561', 'request_id': 'req-c3d29fd9-074d-431d-ac57-923e8bb43ce6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.815542 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] The token cache is being invalidate because project bfa8df4cda7e4dcbb43742d3b9c82561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:37.815825 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.816495 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3d29fd9-074d-431d-ac57-923e8bb43ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:37.818120 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1351/2690] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:37 2022] DELETE /identity/v3/projects/bfa8df4cda7e4dcbb43742d3b9c82561 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:37.823011 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be8b64-085a-40eb-805f-69db8862973e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:37.823314 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be8b64-085a-40eb-805f-69db8862973e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:37.823588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be8b64-085a-40eb-805f-69db8862973e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:37.841379 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-00be8b64-085a-40eb-805f-69db8862973e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:37.842594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-00be8b64-085a-40eb-805f-69db8862973e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:37.902343 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1340/2691] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:37.910875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab2591-2ccf-49f7-8fe1-c9e5162ffaef None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:37.911365 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab2591-2ccf-49f7-8fe1-c9e5162ffaef None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:37.911490 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab2591-2ccf-49f7-8fe1-c9e5162ffaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:37.930817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f4ab2591-2ccf-49f7-8fe1-c9e5162ffaef None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:37.932009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f4ab2591-2ccf-49f7-8fe1-c9e5162ffaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:37.940548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1352/2692] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:37.944752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be8b64-085a-40eb-805f-69db8862973e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:37.954622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be8b64-085a-40eb-805f-69db8862973e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:37.959189 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8146e-5276-4c31-aaeb-331fb842bc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:37.959846 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8146e-5276-4c31-aaeb-331fb842bc91 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:37.960132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8146e-5276-4c31-aaeb-331fb842bc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:37.960414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8146e-5276-4c31-aaeb-331fb842bc91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:37.961150 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8146e-5276-4c31-aaeb-331fb842bc91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:37.961463 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8146e-5276-4c31-aaeb-331fb842bc91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:37.963103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8146e-5276-4c31-aaeb-331fb842bc91 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:37.975191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1341/2693] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:37.979276 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab2591-2ccf-49f7-8fe1-c9e5162ffaef None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:37.989053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab2591-2ccf-49f7-8fe1-c9e5162ffaef None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:37.993630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca801c6d-2562-4130-806f-4df6bd40ed0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:37.994313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca801c6d-2562-4130-806f-4df6bd40ed0d admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:37.994601 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca801c6d-2562-4130-806f-4df6bd40ed0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:37.994877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca801c6d-2562-4130-806f-4df6bd40ed0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:37.995576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca801c6d-2562-4130-806f-4df6bd40ed0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:37.997487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca801c6d-2562-4130-806f-4df6bd40ed0d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:38.005650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:38.024573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1353/2694] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:37 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:38.028923 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8146e-5276-4c31-aaeb-331fb842bc91 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:38.039559 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8146e-5276-4c31-aaeb-331fb842bc91 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:38.051466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:38.100214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf95ae8-810d-40aa-942a-71aa8189cf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:38.100888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95ae8-810d-40aa-942a-71aa8189cf67 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:38.101179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95ae8-810d-40aa-942a-71aa8189cf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:38.101465 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95ae8-810d-40aa-942a-71aa8189cf67 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:38.102196 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf95ae8-810d-40aa-942a-71aa8189cf67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:38.104130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf95ae8-810d-40aa-942a-71aa8189cf67 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:38.124438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1342/2695] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:38 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:38.129096 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca801c6d-2562-4130-806f-4df6bd40ed0d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:38.139647 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca801c6d-2562-4130-806f-4df6bd40ed0d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:38.145539 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e3eaa-8644-46c4-9283-b6756ac0e5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:38.146420 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e3eaa-8644-46c4-9283-b6756ac0e5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:38.146704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e3eaa-8644-46c4-9283-b6756ac0e5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:38.146982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e3eaa-8644-46c4-9283-b6756ac0e5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:38.147786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e3eaa-8644-46c4-9283-b6756ac0e5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:38.149377 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e3eaa-8644-46c4-9283-b6756ac0e5b2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:38.160982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1354/2696] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:38.165024 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf95ae8-810d-40aa-942a-71aa8189cf67 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:38.175790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf95ae8-810d-40aa-942a-71aa8189cf67 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:38.180758 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db190cc8-2f21-45e2-8014-ddd53f63e842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:38.181435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190cc8-2f21-45e2-8014-ddd53f63e842 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:38.181724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190cc8-2f21-45e2-8014-ddd53f63e842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:38.182257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190cc8-2f21-45e2-8014-ddd53f63e842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:38.182776 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db190cc8-2f21-45e2-8014-ddd53f63e842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:38.184432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db190cc8-2f21-45e2-8014-ddd53f63e842 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:38.195278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1343/2697] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:38.200665 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e3eaa-8644-46c4-9283-b6756ac0e5b2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:38.212886 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e3eaa-8644-46c4-9283-b6756ac0e5b2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:38.219237 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac64b8-c555-4ec7-91d3-5134b8f3b104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:38.220634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac64b8-c555-4ec7-91d3-5134b8f3b104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:38.220952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac64b8-c555-4ec7-91d3-5134b8f3b104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:38.221352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac64b8-c555-4ec7-91d3-5134b8f3b104 admin admin] PATH_INFO: `/v3/projects/e803814775d740a1903e4f83904180f8/users/72012d0e4ee44f198851b43fa21e6562/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:38.244979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac64b8-c555-4ec7-91d3-5134b8f3b104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e803814775d740a1903e4f83904180f8, user_id=72012d0e4ee44f198851b43fa21e6562, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:38.247682 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac64b8-c555-4ec7-91d3-5134b8f3b104 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:38.261979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1355/2698] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:38 2022] PUT /identity/v3/projects/e803814775d740a1903e4f83904180f8/users/72012d0e4ee44f198851b43fa21e6562/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:38.268257 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df87be-6557-4955-bea8-20c3fc994642 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:38.268553 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df87be-6557-4955-bea8-20c3fc994642 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:38.268834 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df87be-6557-4955-bea8-20c3fc994642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:38.288217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-72df87be-6557-4955-bea8-20c3fc994642 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:38.289408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-72df87be-6557-4955-bea8-20c3fc994642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:38.353367 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1344/2699] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:38.363761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac64b8-c555-4ec7-91d3-5134b8f3b104 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:38.365794 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a2ac64b8-c555-4ec7-91d3-5134b8f3b104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:38.452630 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac64b8-c555-4ec7-91d3-5134b8f3b104 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:38.457079 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4f5ef-af7e-4412-bb13-60b616f55606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:38.457742 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4f5ef-af7e-4412-bb13-60b616f55606 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:38.458060 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4f5ef-af7e-4412-bb13-60b616f55606 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:38.458343 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4f5ef-af7e-4412-bb13-60b616f55606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:38.470628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4f5ef-af7e-4412-bb13-60b616f55606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:38.472091 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4f5ef-af7e-4412-bb13-60b616f55606 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:38.483641 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1356/2700] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:39.132580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701062c-5837-4d4e-ab82-0649ae2034d6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:39.133096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701062c-5837-4d4e-ab82-0649ae2034d6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:39.133215 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701062c-5837-4d4e-ab82-0649ae2034d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:39.161449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5701062c-5837-4d4e-ab82-0649ae2034d6 None None] MFA Rules not processed for user `72012d0e4ee44f198851b43fa21e6562`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:39.162771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5701062c-5837-4d4e-ab82-0649ae2034d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:39.231926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1345/2701] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:39.236479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4f5ef-af7e-4412-bb13-60b616f55606 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:39.245993 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4f5ef-af7e-4412-bb13-60b616f55606 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:39.250515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d4443-6e38-4a0c-a12a-2cb202e98cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:39.251203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d4443-6e38-4a0c-a12a-2cb202e98cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:39.251485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d4443-6e38-4a0c-a12a-2cb202e98cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:39.251762 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d4443-6e38-4a0c-a12a-2cb202e98cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:39.252448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d4443-6e38-4a0c-a12a-2cb202e98cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:39.254428 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d4443-6e38-4a0c-a12a-2cb202e98cb5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:39.280167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1357/2702] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:39 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:39.284658 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701062c-5837-4d4e-ab82-0649ae2034d6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:39.295206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701062c-5837-4d4e-ab82-0649ae2034d6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:39.315727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:39.354899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9cd8b-f755-4db6-9ad4-c09e2e89a984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:39.355579 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9cd8b-f755-4db6-9ad4-c09e2e89a984 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:39.355862 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9cd8b-f755-4db6-9ad4-c09e2e89a984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:39.356141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9cd8b-f755-4db6-9ad4-c09e2e89a984 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:39.356841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9cd8b-f755-4db6-9ad4-c09e2e89a984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:39.358793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9cd8b-f755-4db6-9ad4-c09e2e89a984 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:39.377679 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1346/2703] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:39 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:39.381782 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d4443-6e38-4a0c-a12a-2cb202e98cb5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:39.393456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d4443-6e38-4a0c-a12a-2cb202e98cb5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:39.398065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff606d1-0b49-4a11-a015-70c548b576bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:39.398731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff606d1-0b49-4a11-a015-70c548b576bc admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:39.399017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff606d1-0b49-4a11-a015-70c548b576bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:39.399296 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff606d1-0b49-4a11-a015-70c548b576bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:39.400024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff606d1-0b49-4a11-a015-70c548b576bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:39.401614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff606d1-0b49-4a11-a015-70c548b576bc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:39.414316 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1358/2704] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:39.419628 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9cd8b-f755-4db6-9ad4-c09e2e89a984 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:39.430231 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9cd8b-f755-4db6-9ad4-c09e2e89a984 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:39.434859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195180e8-37eb-492c-aba1-e1db974cda1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:39.435723 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195180e8-37eb-492c-aba1-e1db974cda1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:39.436008 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195180e8-37eb-492c-aba1-e1db974cda1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:39.436286 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195180e8-37eb-492c-aba1-e1db974cda1b admin admin] PATH_INFO: `/v3/projects/ab0b6879f17240459616b055c9f8f05e/users/9424c59c8c4a40b19a12cbfc95d831d0/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:39.451469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195180e8-37eb-492c-aba1-e1db974cda1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0b6879f17240459616b055c9f8f05e, user_id=9424c59c8c4a40b19a12cbfc95d831d0, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:39.454765 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195180e8-37eb-492c-aba1-e1db974cda1b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:39.469956 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1347/2705] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:39 2022] PUT /identity/v3/projects/ab0b6879f17240459616b055c9f8f05e/users/9424c59c8c4a40b19a12cbfc95d831d0/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:38:39.473955 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff606d1-0b49-4a11-a015-70c548b576bc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:39.484830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff606d1-0b49-4a11-a015-70c548b576bc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:39.543724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa186b-62d1-4ce4-987c-859dde4c9ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:39.544399 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa186b-62d1-4ce4-987c-859dde4c9ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:39.544686 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa186b-62d1-4ce4-987c-859dde4c9ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:39.544952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa186b-62d1-4ce4-987c-859dde4c9ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:39.545678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa186b-62d1-4ce4-987c-859dde4c9ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:39.548006 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa186b-62d1-4ce4-987c-859dde4c9ca3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:39.561660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1359/2706] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:39.563941 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195180e8-37eb-492c-aba1-e1db974cda1b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:39.574634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195180e8-37eb-492c-aba1-e1db974cda1b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:39.582104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172c342-a4a8-489b-b367-b365c6b07531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:39.582104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172c342-a4a8-489b-b367-b365c6b07531 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:39.582104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172c342-a4a8-489b-b367-b365c6b07531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:39.582104 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172c342-a4a8-489b-b367-b365c6b07531 admin admin] PATH_INFO: `/v3/projects/ab0b6879f17240459616b055c9f8f05e/users/9424c59c8c4a40b19a12cbfc95d831d0/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:39.583437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172c342-a4a8-489b-b367-b365c6b07531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0b6879f17240459616b055c9f8f05e, user_id=9424c59c8c4a40b19a12cbfc95d831d0, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:39.586107 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172c342-a4a8-489b-b367-b365c6b07531 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:39.597240 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1348/2707] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:39 2022] PUT /identity/v3/projects/ab0b6879f17240459616b055c9f8f05e/users/9424c59c8c4a40b19a12cbfc95d831d0/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:40.265554 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab594c5-5e45-4577-8c30-50da6ee681f1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:40.266147 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab594c5-5e45-4577-8c30-50da6ee681f1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:40.266330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab594c5-5e45-4577-8c30-50da6ee681f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:40.295596 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cab594c5-5e45-4577-8c30-50da6ee681f1 None None] MFA Rules not processed for user `9424c59c8c4a40b19a12cbfc95d831d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:40.297609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cab594c5-5e45-4577-8c30-50da6ee681f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:40.309984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:40.382659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1360/2708] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:40.395143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172c342-a4a8-489b-b367-b365c6b07531 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:40.397066 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1172c342-a4a8-489b-b367-b365c6b07531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:40.478842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172c342-a4a8-489b-b367-b365c6b07531 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:40.482476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5de17-bf6a-4094-ae65-17b7ddd2e491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:40.483122 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5de17-bf6a-4094-ae65-17b7ddd2e491 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:40.483403 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5de17-bf6a-4094-ae65-17b7ddd2e491 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:40.483679 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5de17-bf6a-4094-ae65-17b7ddd2e491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:40.539862 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5de17-bf6a-4094-ae65-17b7ddd2e491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:40.541210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5de17-bf6a-4094-ae65-17b7ddd2e491 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:40.550179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1349/2709] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:43.832881 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab594c5-5e45-4577-8c30-50da6ee681f1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:43.834350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cab594c5-5e45-4577-8c30-50da6ee681f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:43.899914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab594c5-5e45-4577-8c30-50da6ee681f1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:43.904752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:43.905664 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:43.905967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:43.906882 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] PATH_INFO: `/v3/users/687e102b71a342c2ade6050106540cc9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:43.908137 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=687e102b71a342c2ade6050106540cc9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:43.910098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:43.959205 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '687e102b71a342c2ade6050106540cc9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:43.972166 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '687e102b71a342c2ade6050106540cc9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:43.979200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '687e102b71a342c2ade6050106540cc9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:43.979914 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '687e102b71a342c2ade6050106540cc9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:43.980548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '687e102b71a342c2ade6050106540cc9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:43.994138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1361/2710] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:43 2022] DELETE /identity/v3/users/687e102b71a342c2ade6050106540cc9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:43.998588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38339485-3274-445a-aa7c-75d08c2b801d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:43.999063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38339485-3274-445a-aa7c-75d08c2b801d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:43.999332 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38339485-3274-445a-aa7c-75d08c2b801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:44.002165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:44.003500 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:44.018269 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-38339485-3274-445a-aa7c-75d08c2b801d None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:44.019540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-38339485-3274-445a-aa7c-75d08c2b801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:44.025498 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:44.032160 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:44.067952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5357c4b-7b22-4765-bb16-0d07e79ccff7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:44.071953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad073a-a396-4205-b407-5a76c4247d5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:44.072618 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad073a-a396-4205-b407-5a76c4247d5d service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:44.072905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad073a-a396-4205-b407-5a76c4247d5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:44.073189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad073a-a396-4205-b407-5a76c4247d5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:44.075010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8cad073a-a396-4205-b407-5a76c4247d5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:44.085880 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1350/2711] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:44.096113 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38339485-3274-445a-aa7c-75d08c2b801d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:44.097575 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-38339485-3274-445a-aa7c-75d08c2b801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:44.141485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad073a-a396-4205-b407-5a76c4247d5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:44.142981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad073a-a396-4205-b407-5a76c4247d5d service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:44.152775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1362/2712] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:44.156091 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38339485-3274-445a-aa7c-75d08c2b801d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:44.160766 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6bedb-2ee4-440b-afa1-06a08be319f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:44.161415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6bedb-2ee4-440b-afa1-06a08be319f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:44.161703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6bedb-2ee4-440b-afa1-06a08be319f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:44.162053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6bedb-2ee4-440b-afa1-06a08be319f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:44.171474 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6bedb-2ee4-440b-afa1-06a08be319f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:44.174229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6bedb-2ee4-440b-afa1-06a08be319f8 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:44.184183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1351/2713] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:38:44.481313 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad073a-a396-4205-b407-5a76c4247d5d service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:44.482830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8cad073a-a396-4205-b407-5a76c4247d5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:44.497213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad073a-a396-4205-b407-5a76c4247d5d service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:44.502101 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:44.503003 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:44.503290 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:44.503572 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] PATH_INFO: `/v3/projects/f0dee734971f432e93f1d860790ab325` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:44.504771 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0dee734971f432e93f1d860790ab325)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:44.506723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:44.560880 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0dee734971f432e93f1d860790ab325', 'request_id': 'req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:44.568529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0dee734971f432e93f1d860790ab325', 'request_id': 'req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:44.575844 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0dee734971f432e93f1d860790ab325', 'request_id': 'req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:44.590394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] The token cache is being invalidate because project f0dee734971f432e93f1d860790ab325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:44.590708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:44.591568 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b2f94b8-eb40-4bed-b0bb-02c73a02aacb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:44.593393 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1363/2714] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:44 2022] DELETE /identity/v3/projects/f0dee734971f432e93f1d860790ab325 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:44.599045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157283cf-8975-4596-9406-584c6fd9b51a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:44.599339 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157283cf-8975-4596-9406-584c6fd9b51a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:44.599622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157283cf-8975-4596-9406-584c6fd9b51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:44.620243 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-157283cf-8975-4596-9406-584c6fd9b51a None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:44.622345 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-157283cf-8975-4596-9406-584c6fd9b51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:44.700431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1352/2715] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:44.756534 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b5688-4195-499c-87ca-e85dd1890373 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:44.756825 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b5688-4195-499c-87ca-e85dd1890373 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:44.757101 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b5688-4195-499c-87ca-e85dd1890373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:44.776525 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c44b5688-4195-499c-87ca-e85dd1890373 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:44.777730 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c44b5688-4195-499c-87ca-e85dd1890373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:44.787083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1364/2716] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:44.791355 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157283cf-8975-4596-9406-584c6fd9b51a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:44.810185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157283cf-8975-4596-9406-584c6fd9b51a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:44.810185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04e52-e70a-4b1c-802e-debf27d4e9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:44.810185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04e52-e70a-4b1c-802e-debf27d4e9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:44.810185 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04e52-e70a-4b1c-802e-debf27d4e9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:44.810589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04e52-e70a-4b1c-802e-debf27d4e9d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:44.810589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a04e52-e70a-4b1c-802e-debf27d4e9d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:44.810589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a04e52-e70a-4b1c-802e-debf27d4e9d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:44.810589 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a04e52-e70a-4b1c-802e-debf27d4e9d4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:44.826035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1353/2717] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:44.828739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b5688-4195-499c-87ca-e85dd1890373 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:44.837982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b5688-4195-499c-87ca-e85dd1890373 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:44.842950 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b96bec-2516-4233-8175-d5d281898ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:44.843548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b96bec-2516-4233-8175-d5d281898ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:44.843892 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b96bec-2516-4233-8175-d5d281898ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:44.844204 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b96bec-2516-4233-8175-d5d281898ba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:44.844908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b96bec-2516-4233-8175-d5d281898ba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:44.847017 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b96bec-2516-4233-8175-d5d281898ba0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:44.872928 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1365/2718] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:44 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:44.877731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04e52-e70a-4b1c-802e-debf27d4e9d4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:44.888781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04e52-e70a-4b1c-802e-debf27d4e9d4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:44.938928 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:44.957124 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a185cf6-a8ed-4af8-abef-8f12bf4f3800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:44.957833 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a185cf6-a8ed-4af8-abef-8f12bf4f3800 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:44.958273 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a185cf6-a8ed-4af8-abef-8f12bf4f3800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:44.958556 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a185cf6-a8ed-4af8-abef-8f12bf4f3800 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:44.959253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a185cf6-a8ed-4af8-abef-8f12bf4f3800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:44.961154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a185cf6-a8ed-4af8-abef-8f12bf4f3800 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:44.985822 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1354/2719] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:44 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:44.989826 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b96bec-2516-4233-8175-d5d281898ba0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:45.000671 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b96bec-2516-4233-8175-d5d281898ba0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:45.005356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388d5d4-aac8-4506-935a-a441dd4a8be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:45.006081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388d5d4-aac8-4506-935a-a441dd4a8be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:45.006368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388d5d4-aac8-4506-935a-a441dd4a8be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:45.006645 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388d5d4-aac8-4506-935a-a441dd4a8be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:45.007376 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388d5d4-aac8-4506-935a-a441dd4a8be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:45.008960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388d5d4-aac8-4506-935a-a441dd4a8be4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:45.020082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1366/2720] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:45.024015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a185cf6-a8ed-4af8-abef-8f12bf4f3800 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:45.034389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a185cf6-a8ed-4af8-abef-8f12bf4f3800 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:45.039014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee981d-c45d-49fb-9db4-a367466068c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:45.039673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee981d-c45d-49fb-9db4-a367466068c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:45.039958 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee981d-c45d-49fb-9db4-a367466068c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:45.040237 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee981d-c45d-49fb-9db4-a367466068c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:45.040996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee981d-c45d-49fb-9db4-a367466068c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:45.042649 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee981d-c45d-49fb-9db4-a367466068c8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:45.055768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1355/2721] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:45.059737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388d5d4-aac8-4506-935a-a441dd4a8be4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:45.070580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388d5d4-aac8-4506-935a-a441dd4a8be4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:45.075197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6573df-90cf-40d5-890e-ce026189fd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:45.076082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6573df-90cf-40d5-890e-ce026189fd8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:45.076368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6573df-90cf-40d5-890e-ce026189fd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:45.076675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6573df-90cf-40d5-890e-ce026189fd8c admin admin] PATH_INFO: `/v3/projects/9ba482e050244bc6a359090a616658ec/users/756c74602c4d4bafade01c406ffd6afc/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:45.092608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6573df-90cf-40d5-890e-ce026189fd8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ba482e050244bc6a359090a616658ec, user_id=756c74602c4d4bafade01c406ffd6afc, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:45.095359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6573df-90cf-40d5-890e-ce026189fd8c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:45.097508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:45.110799 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1367/2722] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:45 2022] PUT /identity/v3/projects/9ba482e050244bc6a359090a616658ec/users/756c74602c4d4bafade01c406ffd6afc/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:45.116387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3c33-606b-49c7-b46d-40dbf04741e2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:45.116680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3c33-606b-49c7-b46d-40dbf04741e2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:45.116963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3c33-606b-49c7-b46d-40dbf04741e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:45.135375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3e8b3c33-606b-49c7-b46d-40dbf04741e2 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:45.136612 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3e8b3c33-606b-49c7-b46d-40dbf04741e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:45.199683 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1356/2723] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:45.209779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6573df-90cf-40d5-890e-ce026189fd8c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:45.211533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5b6573df-90cf-40d5-890e-ce026189fd8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:45.272167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6573df-90cf-40d5-890e-ce026189fd8c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:45.276007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36885c3c-061b-40ae-a8cc-4798e15bb7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:45.276659 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36885c3c-061b-40ae-a8cc-4798e15bb7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:45.276947 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36885c3c-061b-40ae-a8cc-4798e15bb7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:45.277232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36885c3c-061b-40ae-a8cc-4798e15bb7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:45.286595 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36885c3c-061b-40ae-a8cc-4798e15bb7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:45.288833 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36885c3c-061b-40ae-a8cc-4798e15bb7d3 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:45.300461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1368/2724] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:47.521678 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b3c33-606b-49c7-b46d-40dbf04741e2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:47.523763 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3e8b3c33-606b-49c7-b46d-40dbf04741e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:47.537922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b3c33-606b-49c7-b46d-40dbf04741e2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:47.542904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:47.543794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:47.544080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:47.544496 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] PATH_INFO: `/v3/users/bd7bb79340404274a85e71bb8fb5997e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:47.545749 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7bb79340404274a85e71bb8fb5997e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:47.549477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:47.608393 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7bb79340404274a85e71bb8fb5997e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:47.609359 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7bb79340404274a85e71bb8fb5997e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:47.619798 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7bb79340404274a85e71bb8fb5997e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:47.634948 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7bb79340404274a85e71bb8fb5997e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:47.635742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7bb79340404274a85e71bb8fb5997e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:47.652404 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1357/2725] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:47 2022] DELETE /identity/v3/users/bd7bb79340404274a85e71bb8fb5997e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:47.657803 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20b4ca-385a-47af-880e-267ffbb5415e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:47.658314 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20b4ca-385a-47af-880e-267ffbb5415e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:47.658783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20b4ca-385a-47af-880e-267ffbb5415e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:47.686978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9b20b4ca-385a-47af-880e-267ffbb5415e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:47.688907 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9b20b4ca-385a-47af-880e-267ffbb5415e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:47.825178 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1369/2726] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:47.835394 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:47.837166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:47.900374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:47.952195 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ee475-e178-4ae3-91c9-fe2a99685a50 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:47.960220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ff8fa-f05a-4731-9c0b-5619530e4cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:47.961252 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff8fa-f05a-4731-9c0b-5619530e4cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:47.961584 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff8fa-f05a-4731-9c0b-5619530e4cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:47.961986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff8fa-f05a-4731-9c0b-5619530e4cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:47.975866 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ff8fa-f05a-4731-9c0b-5619530e4cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:47.977391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ff8fa-f05a-4731-9c0b-5619530e4cc5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:47.994023 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1358/2727] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:38:48.906842 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20b4ca-385a-47af-880e-267ffbb5415e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:48.909961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9b20b4ca-385a-47af-880e-267ffbb5415e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:48.942532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20b4ca-385a-47af-880e-267ffbb5415e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:48.952707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:48.955364 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:48.955852 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:48.956818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] PATH_INFO: `/v3/projects/7738548b6bed49abadd5556f90281920` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:48.958650 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7738548b6bed49abadd5556f90281920)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:48.962340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:49.034931 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7738548b6bed49abadd5556f90281920', 'request_id': 'req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.056046 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7738548b6bed49abadd5556f90281920', 'request_id': 'req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.066093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7738548b6bed49abadd5556f90281920', 'request_id': 'req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.081001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] The token cache is being invalidate because project 7738548b6bed49abadd5556f90281920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:49.081289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.082177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.087317 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1370/2728] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:48 2022] DELETE /identity/v3/projects/7738548b6bed49abadd5556f90281920 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:49.089751 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ff8fa-f05a-4731-9c0b-5619530e4cc5 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:49.102513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ff8fa-f05a-4731-9c0b-5619530e4cc5 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:49.169246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be981a71-7d1b-4023-a467-a44f6a1e47a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:49.170147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be981a71-7d1b-4023-a467-a44f6a1e47a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:49.170428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be981a71-7d1b-4023-a467-a44f6a1e47a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:49.170699 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be981a71-7d1b-4023-a467-a44f6a1e47a2 admin admin] PATH_INFO: `/v3/users/fc607670b44e4d9197655743d6285adf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:49.171896 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be981a71-7d1b-4023-a467-a44f6a1e47a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc607670b44e4d9197655743d6285adf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:49.173859 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be981a71-7d1b-4023-a467-a44f6a1e47a2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:49.232346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-be981a71-7d1b-4023-a467-a44f6a1e47a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc607670b44e4d9197655743d6285adf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.234305 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-be981a71-7d1b-4023-a467-a44f6a1e47a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc607670b44e4d9197655743d6285adf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.255265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-be981a71-7d1b-4023-a467-a44f6a1e47a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc607670b44e4d9197655743d6285adf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.278930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-be981a71-7d1b-4023-a467-a44f6a1e47a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc607670b44e4d9197655743d6285adf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.279971 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-be981a71-7d1b-4023-a467-a44f6a1e47a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc607670b44e4d9197655743d6285adf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.295319 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1359/2729] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:49 2022] DELETE /identity/v3/users/fc607670b44e4d9197655743d6285adf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:49.663815 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:49.666830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:49.693669 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c2f6b-531a-43eb-a5df-aedf6d52e76d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:49.732634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:49.838868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:49.839811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:49.840037 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:49.840312 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] PATH_INFO: `/v3/projects/a13e784219d948f6a7d6be6df8af6c97` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:49.842273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a13e784219d948f6a7d6be6df8af6c97)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:49.844638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:49.908246 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307cf5-448f-4327-babe-5ad635bd3fe6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:49.910111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307cf5-448f-4327-babe-5ad635bd3fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:49.910801 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307cf5-448f-4327-babe-5ad635bd3fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:49.941197 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a13e784219d948f6a7d6be6df8af6c97', 'request_id': 'req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.959570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a13e784219d948f6a7d6be6df8af6c97', 'request_id': 'req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.964275 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a5307cf5-448f-4327-babe-5ad635bd3fe6 None None] MFA Rules not processed for user `756c74602c4d4bafade01c406ffd6afc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:49.965720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a5307cf5-448f-4327-babe-5ad635bd3fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:49.970202 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a13e784219d948f6a7d6be6df8af6c97', 'request_id': 'req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.987441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] The token cache is being invalidate because project a13e784219d948f6a7d6be6df8af6c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:38:49.987606 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.988469 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2a6b5500-4ccb-4f07-b96a-11fcd486ff11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:49.993423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1371/2730] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:38:49 2022] DELETE /identity/v3/projects/a13e784219d948f6a7d6be6df8af6c97 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:49.995763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bc6de-5d45-4068-b0e2-367aaa65ea0f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:49.996057 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bc6de-5d45-4068-b0e2-367aaa65ea0f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:49.996338 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bc6de-5d45-4068-b0e2-367aaa65ea0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.036065 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5c1bc6de-5d45-4068-b0e2-367aaa65ea0f None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:50.037375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5c1bc6de-5d45-4068-b0e2-367aaa65ea0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:50.052690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:50.090860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1360/2731] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:50.095815 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307cf5-448f-4327-babe-5ad635bd3fe6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.097214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a5307cf5-448f-4327-babe-5ad635bd3fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:50.153501 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1372/2732] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:50.161700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4224f-18c9-4543-8a2e-6a52d6abb42b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.162018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4224f-18c9-4543-8a2e-6a52d6abb42b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.162307 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4224f-18c9-4543-8a2e-6a52d6abb42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.190351 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-09f4224f-18c9-4543-8a2e-6a52d6abb42b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:50.190767 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307cf5-448f-4327-babe-5ad635bd3fe6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.191610 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-09f4224f-18c9-4543-8a2e-6a52d6abb42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:50.196990 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394e367-e8e5-4d88-8966-5d5328a62ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.197703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394e367-e8e5-4d88-8966-5d5328a62ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.198020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394e367-e8e5-4d88-8966-5d5328a62ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.198309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394e367-e8e5-4d88-8966-5d5328a62ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.199016 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394e367-e8e5-4d88-8966-5d5328a62ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.200018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1373/2733] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:50.201074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394e367-e8e5-4d88-8966-5d5328a62ebc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.205071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4224f-18c9-4543-8a2e-6a52d6abb42b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.215996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4224f-18c9-4543-8a2e-6a52d6abb42b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.222817 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a8f99-fe42-4006-a622-2fb41e10ee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.223663 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a8f99-fe42-4006-a622-2fb41e10ee7f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.224071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a8f99-fe42-4006-a622-2fb41e10ee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.224470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a8f99-fe42-4006-a622-2fb41e10ee7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.225288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a8f99-fe42-4006-a622-2fb41e10ee7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:38:50.225597 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a8f99-fe42-4006-a622-2fb41e10ee7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.227273 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a8f99-fe42-4006-a622-2fb41e10ee7f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.241103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1361/2734] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:50 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:38:50.242056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394e367-e8e5-4d88-8966-5d5328a62ebc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.244162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1374/2735] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:38:50.249576 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a8f99-fe42-4006-a622-2fb41e10ee7f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.261577 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a8f99-fe42-4006-a622-2fb41e10ee7f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.305978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:50.343748 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58ae35-3852-462c-b95e-08f2ea0e2c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.345125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58ae35-3852-462c-b95e-08f2ea0e2c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.345415 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58ae35-3852-462c-b95e-08f2ea0e2c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.345699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58ae35-3852-462c-b95e-08f2ea0e2c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.346613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58ae35-3852-462c-b95e-08f2ea0e2c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.348024 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394e367-e8e5-4d88-8966-5d5328a62ebc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.349913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58ae35-3852-462c-b95e-08f2ea0e2c23 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.352991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e9368-6920-467e-b3e0-f94e2a26c9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.353752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9368-6920-467e-b3e0-f94e2a26c9b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.354153 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9368-6920-467e-b3e0-f94e2a26c9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.354513 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9368-6920-467e-b3e0-f94e2a26c9b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.355312 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e9368-6920-467e-b3e0-f94e2a26c9b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.357434 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e9368-6920-467e-b3e0-f94e2a26c9b0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.386126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1362/2736] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:50 2022] POST /identity/v3/users => generated 299 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:50.386335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1375/2737] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:50 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:50.389223 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e9368-6920-467e-b3e0-f94e2a26c9b0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.389374 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58ae35-3852-462c-b95e-08f2ea0e2c23 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.401307 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e9368-6920-467e-b3e0-f94e2a26c9b0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.456192 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58ae35-3852-462c-b95e-08f2ea0e2c23 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.460991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d3c32-7b7c-49c7-b533-d459be329828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.461672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d3c32-7b7c-49c7-b533-d459be329828 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.461989 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d3c32-7b7c-49c7-b533-d459be329828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.462278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d3c32-7b7c-49c7-b533-d459be329828 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.462402 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd091-f999-4515-992d-ca28b97582de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.462968 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd091-f999-4515-992d-ca28b97582de admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.463084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d3c32-7b7c-49c7-b533-d459be329828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.463258 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd091-f999-4515-992d-ca28b97582de admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.463540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd091-f999-4515-992d-ca28b97582de admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.464276 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd091-f999-4515-992d-ca28b97582de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.464922 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d3c32-7b7c-49c7-b533-d459be329828 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.465867 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd091-f999-4515-992d-ca28b97582de admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.467214 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:50.481662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1363/2738] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:50.482580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd091-f999-4515-992d-ca28b97582de admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.488774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1376/2739] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:50 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:50.489889 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d3c32-7b7c-49c7-b533-d459be329828 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.493283 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd091-f999-4515-992d-ca28b97582de admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.497961 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa6cbf-e4e2-464c-8248-a4abb7ee95bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.498853 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa6cbf-e4e2-464c-8248-a4abb7ee95bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.499136 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa6cbf-e4e2-464c-8248-a4abb7ee95bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.499413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa6cbf-e4e2-464c-8248-a4abb7ee95bf admin admin] PATH_INFO: `/v3/projects/f58c7912c6b24db6b34dde825f9e6c04/users/941e6b2a346d4a93ad614de89460d7b2/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.499676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d3c32-7b7c-49c7-b533-d459be329828 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.504870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c665a83-3f81-48fc-a371-fc03b064c6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.505526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c665a83-3f81-48fc-a371-fc03b064c6bd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.505813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c665a83-3f81-48fc-a371-fc03b064c6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.506123 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c665a83-3f81-48fc-a371-fc03b064c6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.506861 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c665a83-3f81-48fc-a371-fc03b064c6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.508458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c665a83-3f81-48fc-a371-fc03b064c6bd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.515160 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa6cbf-e4e2-464c-8248-a4abb7ee95bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58c7912c6b24db6b34dde825f9e6c04, user_id=941e6b2a346d4a93ad614de89460d7b2, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.517827 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa6cbf-e4e2-464c-8248-a4abb7ee95bf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.518573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1377/2740] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:50.522520 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c665a83-3f81-48fc-a371-fc03b064c6bd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.532356 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c665a83-3f81-48fc-a371-fc03b064c6bd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.532561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1364/2741] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:50 2022] PUT /identity/v3/projects/f58c7912c6b24db6b34dde825f9e6c04/users/941e6b2a346d4a93ad614de89460d7b2/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:50.536414 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa6cbf-e4e2-464c-8248-a4abb7ee95bf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.595459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe72758-a2c9-4af0-8e57-42a4c1ba1057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.596165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe72758-a2c9-4af0-8e57-42a4c1ba1057 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.596452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe72758-a2c9-4af0-8e57-42a4c1ba1057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.596733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe72758-a2c9-4af0-8e57-42a4c1ba1057 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.597486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe72758-a2c9-4af0-8e57-42a4c1ba1057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.599156 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe72758-a2c9-4af0-8e57-42a4c1ba1057 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.605186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa6cbf-e4e2-464c-8248-a4abb7ee95bf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.613359 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1378/2742] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:38:50.613769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08acb6-7145-4d63-bd31-8792d5b80350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.614477 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe72758-a2c9-4af0-8e57-42a4c1ba1057 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.614906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08acb6-7145-4d63-bd31-8792d5b80350 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.615452 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08acb6-7145-4d63-bd31-8792d5b80350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.615920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08acb6-7145-4d63-bd31-8792d5b80350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.616677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08acb6-7145-4d63-bd31-8792d5b80350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.618376 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08acb6-7145-4d63-bd31-8792d5b80350 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.624702 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe72758-a2c9-4af0-8e57-42a4c1ba1057 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.631012 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1365/2743] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:38:50.631777 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1c4c4-7701-4015-b925-426041942ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.632936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1c4c4-7701-4015-b925-426041942ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.633227 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1c4c4-7701-4015-b925-426041942ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.633511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1c4c4-7701-4015-b925-426041942ebe admin admin] PATH_INFO: `/v3/projects/6ab91c9f71e443dd8ffca5f13945f0d5/users/852eb64000644b6d990ffd1ddad6e699/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.635213 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08acb6-7145-4d63-bd31-8792d5b80350 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.645879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08acb6-7145-4d63-bd31-8792d5b80350 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.650048 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1c4c4-7701-4015-b925-426041942ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ab91c9f71e443dd8ffca5f13945f0d5, user_id=852eb64000644b6d990ffd1ddad6e699, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.650881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93904f-b712-479d-a236-17c6190e6a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.651888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93904f-b712-479d-a236-17c6190e6a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.652168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93904f-b712-479d-a236-17c6190e6a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.652484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93904f-b712-479d-a236-17c6190e6a1c admin admin] PATH_INFO: `/v3/projects/f58c7912c6b24db6b34dde825f9e6c04/users/941e6b2a346d4a93ad614de89460d7b2/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.652681 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1c4c4-7701-4015-b925-426041942ebe admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.654840 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93904f-b712-479d-a236-17c6190e6a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58c7912c6b24db6b34dde825f9e6c04, user_id=941e6b2a346d4a93ad614de89460d7b2, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.658385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93904f-b712-479d-a236-17c6190e6a1c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.669986 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1379/2744] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:50 2022] PUT /identity/v3/projects/6ab91c9f71e443dd8ffca5f13945f0d5/users/852eb64000644b6d990ffd1ddad6e699/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:50.678407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48906c8b-5b57-4109-878b-ad810a6b6726 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.678824 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48906c8b-5b57-4109-878b-ad810a6b6726 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.679188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48906c8b-5b57-4109-878b-ad810a6b6726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.679780 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1366/2745] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:50 2022] PUT /identity/v3/projects/f58c7912c6b24db6b34dde825f9e6c04/users/941e6b2a346d4a93ad614de89460d7b2/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:50.693497 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:50.711446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-48906c8b-5b57-4109-878b-ad810a6b6726 None None] MFA Rules not processed for user `852eb64000644b6d990ffd1ddad6e699`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:50.712716 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-48906c8b-5b57-4109-878b-ad810a6b6726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:50.794371 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1380/2746] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:38:50.796542 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93904f-b712-479d-a236-17c6190e6a1c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.868068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93904f-b712-479d-a236-17c6190e6a1c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.873739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed63355-8577-4683-a27b-28ba1366f2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.874446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed63355-8577-4683-a27b-28ba1366f2ec admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.874731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed63355-8577-4683-a27b-28ba1366f2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.875752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed63355-8577-4683-a27b-28ba1366f2ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.876613 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed63355-8577-4683-a27b-28ba1366f2ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:50.878554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed63355-8577-4683-a27b-28ba1366f2ec admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:50.909425 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1367/2747] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:38:50 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:50.913570 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48906c8b-5b57-4109-878b-ad810a6b6726 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:50.931045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48906c8b-5b57-4109-878b-ad810a6b6726 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:50.997053 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b637a-5c49-4eb6-bbb1-c69a9e2df735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:50.997768 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b637a-5c49-4eb6-bbb1-c69a9e2df735 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:50.998127 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b637a-5c49-4eb6-bbb1-c69a9e2df735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:50.998349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b637a-5c49-4eb6-bbb1-c69a9e2df735 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:50.999074 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b637a-5c49-4eb6-bbb1-c69a9e2df735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:51.001602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b637a-5c49-4eb6-bbb1-c69a9e2df735 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:51.024896 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1381/2748] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:38:50 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:38:51.033881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed63355-8577-4683-a27b-28ba1366f2ec admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:51.056186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed63355-8577-4683-a27b-28ba1366f2ec admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:51.065616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7cb0a-042c-46da-9368-f76175113377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:51.065852 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7cb0a-042c-46da-9368-f76175113377 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:51.066242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7cb0a-042c-46da-9368-f76175113377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:51.066913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7cb0a-042c-46da-9368-f76175113377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:51.067488 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7cb0a-042c-46da-9368-f76175113377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:51.074409 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7cb0a-042c-46da-9368-f76175113377 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:51.088998 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1368/2749] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:51.094032 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b637a-5c49-4eb6-bbb1-c69a9e2df735 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:51.106132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b637a-5c49-4eb6-bbb1-c69a9e2df735 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:51.114511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627538a-260b-45d0-a013-cc2686a74a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:51.115814 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627538a-260b-45d0-a013-cc2686a74a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:51.116297 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627538a-260b-45d0-a013-cc2686a74a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:51.116561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627538a-260b-45d0-a013-cc2686a74a27 admin admin] PATH_INFO: `/v3/projects/c784526e06e34f33bfcb39b3bedcffdb/users/2f84d22e3a7d43ab913eb757c28841fa/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:51.140110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627538a-260b-45d0-a013-cc2686a74a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c784526e06e34f33bfcb39b3bedcffdb, user_id=2f84d22e3a7d43ab913eb757c28841fa, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:51.143363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627538a-260b-45d0-a013-cc2686a74a27 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:51.169910 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1382/2750] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:51 2022] PUT /identity/v3/projects/c784526e06e34f33bfcb39b3bedcffdb/users/2f84d22e3a7d43ab913eb757c28841fa/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:51.174694 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7cb0a-042c-46da-9368-f76175113377 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:51.191323 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7cb0a-042c-46da-9368-f76175113377 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:51.231711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:51.281279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a5590-9eb6-402f-84c7-e0de0495c765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:51.283668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a5590-9eb6-402f-84c7-e0de0495c765 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:51.284083 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a5590-9eb6-402f-84c7-e0de0495c765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:51.284469 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a5590-9eb6-402f-84c7-e0de0495c765 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:51.286132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a5590-9eb6-402f-84c7-e0de0495c765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:51.287991 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a5590-9eb6-402f-84c7-e0de0495c765 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:51.313604 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1369/2751] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:38:51.318798 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627538a-260b-45d0-a013-cc2686a74a27 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:51.332234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627538a-260b-45d0-a013-cc2686a74a27 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:51.341961 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f9f61-e861-4e47-91fb-2de6f95b80b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:51.342956 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f9f61-e861-4e47-91fb-2de6f95b80b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:51.343248 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f9f61-e861-4e47-91fb-2de6f95b80b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:51.343589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f9f61-e861-4e47-91fb-2de6f95b80b5 admin admin] PATH_INFO: `/v3/projects/c784526e06e34f33bfcb39b3bedcffdb/users/2f84d22e3a7d43ab913eb757c28841fa/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:51.347098 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f9f61-e861-4e47-91fb-2de6f95b80b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c784526e06e34f33bfcb39b3bedcffdb, user_id=2f84d22e3a7d43ab913eb757c28841fa, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:51.350168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f9f61-e861-4e47-91fb-2de6f95b80b5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:51.370335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1383/2752] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:38:51 2022] PUT /identity/v3/projects/c784526e06e34f33bfcb39b3bedcffdb/users/2f84d22e3a7d43ab913eb757c28841fa/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:38:51.379736 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa3736-5ddb-4734-94d3-8ad9cacb04fa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:51.380068 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa3736-5ddb-4734-94d3-8ad9cacb04fa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:51.380580 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa3736-5ddb-4734-94d3-8ad9cacb04fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:51.440001 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-06fa3736-5ddb-4734-94d3-8ad9cacb04fa None None] MFA Rules not processed for user `2f84d22e3a7d43ab913eb757c28841fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:51.442451 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-06fa3736-5ddb-4734-94d3-8ad9cacb04fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:51.545904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1370/2753] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:51.558634 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f9f61-e861-4e47-91fb-2de6f95b80b5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:51.560778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f65f9f61-e861-4e47-91fb-2de6f95b80b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:51.625362 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:51.630588 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f9f61-e861-4e47-91fb-2de6f95b80b5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:51.635278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38179ea9-02e1-4f23-bf07-61c8ed1d375a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:51.635983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38179ea9-02e1-4f23-bf07-61c8ed1d375a service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:51.637523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38179ea9-02e1-4f23-bf07-61c8ed1d375a service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:51.637523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38179ea9-02e1-4f23-bf07-61c8ed1d375a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:51.715543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38179ea9-02e1-4f23-bf07-61c8ed1d375a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:51.724574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38179ea9-02e1-4f23-bf07-61c8ed1d375a service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:51.752042 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1384/2754] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:55.964572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195697e-6008-453b-8022-373cf7f99b02 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:55.964572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195697e-6008-453b-8022-373cf7f99b02 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:55.964572 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195697e-6008-453b-8022-373cf7f99b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:56.012620 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9195697e-6008-453b-8022-373cf7f99b02 None None] MFA Rules not processed for user `941e6b2a346d4a93ad614de89460d7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:56.013873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9195697e-6008-453b-8022-373cf7f99b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:56.165999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1371/2755] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:38:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:56.181574 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38179ea9-02e1-4f23-bf07-61c8ed1d375a service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:56.184550 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-38179ea9-02e1-4f23-bf07-61c8ed1d375a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:56.280510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38179ea9-02e1-4f23-bf07-61c8ed1d375a service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:56.285801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6d43c-3a3c-4f80-96ba-28fd7585e6c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa2c8be3105a415ead65579dc18ba40d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:56.286528 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6d43c-3a3c-4f80-96ba-28fd7585e6c2 service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:56.286816 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6d43c-3a3c-4f80-96ba-28fd7585e6c2 service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:56.287099 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6d43c-3a3c-4f80-96ba-28fd7585e6c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:56.416767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6d43c-3a3c-4f80-96ba-28fd7585e6c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:56.418270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6d43c-3a3c-4f80-96ba-28fd7585e6c2 service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:56.435168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1385/2756] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:38:56.603495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cbc0b-d8e4-42ab-8c5b-32c7b2a6b085 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:56.603789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cbc0b-d8e4-42ab-8c5b-32c7b2a6b085 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:56.604076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cbc0b-d8e4-42ab-8c5b-32c7b2a6b085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:56.624818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6d43c-3a3c-4f80-96ba-28fd7585e6c2 service glance] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:56.634785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6d43c-3a3c-4f80-96ba-28fd7585e6c2 service glance] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:56.641460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced033a-9c34-4c20-b538-b1ea9699e41e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:56.643363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced033a-9c34-4c20-b538-b1ea9699e41e service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:56.643656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced033a-9c34-4c20-b538-b1ea9699e41e service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:56.644293 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced033a-9c34-4c20-b538-b1ea9699e41e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:56.645999 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0a9cbc0b-d8e4-42ab-8c5b-32c7b2a6b085 None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:38:56.647565 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0a9cbc0b-d8e4-42ab-8c5b-32c7b2a6b085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:56.656354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced033a-9c34-4c20-b538-b1ea9699e41e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:56.658616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced033a-9c34-4c20-b538-b1ea9699e41e service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:56.669433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1386/2757] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:38:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:38:56.671550 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:56.711347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1372/2758] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:38:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:38:56.764905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced033a-9c34-4c20-b538-b1ea9699e41e service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:56.766811 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-eced033a-9c34-4c20-b538-b1ea9699e41e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:56.822390 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:38:56.850436 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced033a-9c34-4c20-b538-b1ea9699e41e service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:56.857446 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a244fec-0a76-431b-ae80-9592f9d158ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:56.858152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a244fec-0a76-431b-ae80-9592f9d158ff service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:56.858456 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a244fec-0a76-431b-ae80-9592f9d158ff service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:56.858741 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a244fec-0a76-431b-ae80-9592f9d158ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:56.872685 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a244fec-0a76-431b-ae80-9592f9d158ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:56.874320 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a244fec-0a76-431b-ae80-9592f9d158ff service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:56.883923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1387/2759] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:38:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:38:59.337873 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cbc0b-d8e4-42ab-8c5b-32c7b2a6b085 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:38:59.339871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0a9cbc0b-d8e4-42ab-8c5b-32c7b2a6b085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:38:59.431692 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cbc0b-d8e4-42ab-8c5b-32c7b2a6b085 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:38:59.438718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:38:59.439474 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:38:59.439913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:38:59.440266 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] PATH_INFO: `/v3/users/72012d0e4ee44f198851b43fa21e6562` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:38:59.442361 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72012d0e4ee44f198851b43fa21e6562)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:38:59.444425 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:38:59.515204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72012d0e4ee44f198851b43fa21e6562'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:59.523109 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72012d0e4ee44f198851b43fa21e6562'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:59.540841 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72012d0e4ee44f198851b43fa21e6562'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:59.560974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72012d0e4ee44f198851b43fa21e6562'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:59.566881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72012d0e4ee44f198851b43fa21e6562'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:38:59.613386 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1373/2760] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:38:59 2022] DELETE /identity/v3/users/72012d0e4ee44f198851b43fa21e6562 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:00.025920 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a244fec-0a76-431b-ae80-9592f9d158ff service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:00.045209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a244fec-0a76-431b-ae80-9592f9d158ff service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:00.188363 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:00.189280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:00.189564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:00.189839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] PATH_INFO: `/v3/projects/e803814775d740a1903e4f83904180f8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:00.195010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e803814775d740a1903e4f83904180f8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:00.197278 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:00.269906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e803814775d740a1903e4f83904180f8', 'request_id': 'req-ed497eb2-bafc-4a38-981c-76151fa3601b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:00.284470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e803814775d740a1903e4f83904180f8', 'request_id': 'req-ed497eb2-bafc-4a38-981c-76151fa3601b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:00.294969 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e803814775d740a1903e4f83904180f8', 'request_id': 'req-ed497eb2-bafc-4a38-981c-76151fa3601b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:00.310564 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] The token cache is being invalidate because project e803814775d740a1903e4f83904180f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:39:00.311111 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:00.311851 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:00.314933 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1388/2761] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:39:00 2022] DELETE /identity/v3/projects/e803814775d740a1903e4f83904180f8 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:00.318747 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:00.324033 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:00.339512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6cd1e-8664-45ad-ae40-e566a79dff89 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:00.395786 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:00.411491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:00.412385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:00.412668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:00.412946 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] PATH_INFO: `/v3/users/9424c59c8c4a40b19a12cbfc95d831d0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:00.414253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9424c59c8c4a40b19a12cbfc95d831d0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:00.416718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:00.489845 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9424c59c8c4a40b19a12cbfc95d831d0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:00.494527 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9424c59c8c4a40b19a12cbfc95d831d0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:00.504419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9424c59c8c4a40b19a12cbfc95d831d0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:00.518265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9424c59c8c4a40b19a12cbfc95d831d0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:00.519045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9424c59c8c4a40b19a12cbfc95d831d0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:00.539326 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1374/2762] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:39:00 2022] DELETE /identity/v3/users/9424c59c8c4a40b19a12cbfc95d831d0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:00.772301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:00.773883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:00.792904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed497eb2-bafc-4a38-981c-76151fa3601b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:00.908653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:00.909551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:00.909843 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:00.910220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] PATH_INFO: `/v3/projects/ab0b6879f17240459616b055c9f8f05e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:00.911368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0b6879f17240459616b055c9f8f05e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:00.913340 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:00.923521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:01.019919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0b6879f17240459616b055c9f8f05e', 'request_id': 'req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:01.030349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0b6879f17240459616b055c9f8f05e', 'request_id': 'req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:01.039034 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0b6879f17240459616b055c9f8f05e', 'request_id': 'req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:01.055848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] The token cache is being invalidate because project ab0b6879f17240459616b055c9f8f05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:39:01.055848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:01.055848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-07b160f9-b12e-4b0c-b09c-ecbc1d509cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:01.057660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1389/2763] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:39:00 2022] DELETE /identity/v3/projects/ab0b6879f17240459616b055c9f8f05e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:39:02.128832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:02.130711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:02.147163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4a62b-5544-463a-9a43-98a99166b036 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:02.219890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:02.220898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:02.221248 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:02.221711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] PATH_INFO: `/v3/users/852eb64000644b6d990ffd1ddad6e699` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:02.223070 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=852eb64000644b6d990ffd1ddad6e699)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:02.225518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:02.285397 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '852eb64000644b6d990ffd1ddad6e699'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:02.286346 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '852eb64000644b6d990ffd1ddad6e699'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:02.298137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '852eb64000644b6d990ffd1ddad6e699'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:02.310737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:02.314837 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '852eb64000644b6d990ffd1ddad6e699'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:02.315616 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '852eb64000644b6d990ffd1ddad6e699'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:02.333978 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1375/2764] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:39:02 2022] DELETE /identity/v3/users/852eb64000644b6d990ffd1ddad6e699 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:02.338432 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6854c-0c8f-4e37-bb5a-2158df5e1b3b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:02.339266 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6854c-0c8f-4e37-bb5a-2158df5e1b3b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:02.339731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6854c-0c8f-4e37-bb5a-2158df5e1b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:02.361526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fcb6854c-0c8f-4e37-bb5a-2158df5e1b3b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:02.363081 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fcb6854c-0c8f-4e37-bb5a-2158df5e1b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:02.438680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1390/2765] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:02.446366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:02.447803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:02.516810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3bb95-a371-48f4-b2f5-bd7ea50625ad admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:02.523368 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97446bbf-39d0-4c35-ae39-0bf74c3f8722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:02.524049 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97446bbf-39d0-4c35-ae39-0bf74c3f8722 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:02.524424 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97446bbf-39d0-4c35-ae39-0bf74c3f8722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:02.524720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97446bbf-39d0-4c35-ae39-0bf74c3f8722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:02.535328 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97446bbf-39d0-4c35-ae39-0bf74c3f8722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:02.538790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97446bbf-39d0-4c35-ae39-0bf74c3f8722 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:02.550673 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1376/2766] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:02.852163 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6854c-0c8f-4e37-bb5a-2158df5e1b3b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:02.853617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fcb6854c-0c8f-4e37-bb5a-2158df5e1b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:02.867444 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6854c-0c8f-4e37-bb5a-2158df5e1b3b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:02.872786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:02.873739 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:02.874051 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:02.874385 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] PATH_INFO: `/v3/projects/6ab91c9f71e443dd8ffca5f13945f0d5` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:02.877155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ab91c9f71e443dd8ffca5f13945f0d5)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:02.879361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:02.944033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ab91c9f71e443dd8ffca5f13945f0d5', 'request_id': 'req-e8751220-ee3e-4fcb-9a20-128dfedbb733'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:02.955963 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ab91c9f71e443dd8ffca5f13945f0d5', 'request_id': 'req-e8751220-ee3e-4fcb-9a20-128dfedbb733'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:02.963203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ab91c9f71e443dd8ffca5f13945f0d5', 'request_id': 'req-e8751220-ee3e-4fcb-9a20-128dfedbb733'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:02.980158 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] The token cache is being invalidate because project 6ab91c9f71e443dd8ffca5f13945f0d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:39:02.980418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:02.981088 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:02.982679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1391/2767] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:39:02 2022] DELETE /identity/v3/projects/6ab91c9f71e443dd8ffca5f13945f0d5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:02.988247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97446bbf-39d0-4c35-ae39-0bf74c3f8722 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:02.998646 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97446bbf-39d0-4c35-ae39-0bf74c3f8722 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:03.071419 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702aab41-570c-46e2-901e-0ca65f0cb9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:03.072377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702aab41-570c-46e2-901e-0ca65f0cb9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:03.072665 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702aab41-570c-46e2-901e-0ca65f0cb9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:03.072947 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702aab41-570c-46e2-901e-0ca65f0cb9c8 admin admin] PATH_INFO: `/v3/users/2f84d22e3a7d43ab913eb757c28841fa` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:03.074199 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702aab41-570c-46e2-901e-0ca65f0cb9c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f84d22e3a7d43ab913eb757c28841fa)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:03.076137 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702aab41-570c-46e2-901e-0ca65f0cb9c8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:03.139064 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-702aab41-570c-46e2-901e-0ca65f0cb9c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f84d22e3a7d43ab913eb757c28841fa'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:03.140088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-702aab41-570c-46e2-901e-0ca65f0cb9c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f84d22e3a7d43ab913eb757c28841fa'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:03.148462 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-702aab41-570c-46e2-901e-0ca65f0cb9c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f84d22e3a7d43ab913eb757c28841fa'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:03.166037 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-702aab41-570c-46e2-901e-0ca65f0cb9c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f84d22e3a7d43ab913eb757c28841fa'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:03.166814 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-702aab41-570c-46e2-901e-0ca65f0cb9c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f84d22e3a7d43ab913eb757c28841fa'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:03.183520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1377/2768] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:39:02 2022] DELETE /identity/v3/users/2f84d22e3a7d43ab913eb757c28841fa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:03.522432 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:03.524566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:03.546763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8751220-ee3e-4fcb-9a20-128dfedbb733 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:03.567332 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:03.620187 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:03.621286 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:03.621573 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:03.621911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] PATH_INFO: `/v3/projects/c784526e06e34f33bfcb39b3bedcffdb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:03.623132 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c784526e06e34f33bfcb39b3bedcffdb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:03.625289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:03.691113 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c784526e06e34f33bfcb39b3bedcffdb', 'request_id': 'req-214ec2f3-912e-4679-9955-b1fcb46927d5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:03.701476 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c784526e06e34f33bfcb39b3bedcffdb', 'request_id': 'req-214ec2f3-912e-4679-9955-b1fcb46927d5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:03.708859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c784526e06e34f33bfcb39b3bedcffdb', 'request_id': 'req-214ec2f3-912e-4679-9955-b1fcb46927d5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:03.720998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] The token cache is being invalidate because project c784526e06e34f33bfcb39b3bedcffdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:39:03.721364 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:03.722233 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-214ec2f3-912e-4679-9955-b1fcb46927d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:03.723871 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1392/2769] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:39:03 2022] DELETE /identity/v3/projects/c784526e06e34f33bfcb39b3bedcffdb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:03.729773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a37de1-d700-46ae-a818-63a07866c66e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:03.730098 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a37de1-d700-46ae-a818-63a07866c66e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:03.730382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a37de1-d700-46ae-a818-63a07866c66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:03.754080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-64a37de1-d700-46ae-a818-63a07866c66e None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:03.755603 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-64a37de1-d700-46ae-a818-63a07866c66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:03.764386 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:03.836643 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1378/2770] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:03.845141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798aecc1-8aa9-4610-8954-df4b1aa6e6ff None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:03.845432 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798aecc1-8aa9-4610-8954-df4b1aa6e6ff None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:03.845712 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798aecc1-8aa9-4610-8954-df4b1aa6e6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:03.865404 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-798aecc1-8aa9-4610-8954-df4b1aa6e6ff None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:03.866652 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-798aecc1-8aa9-4610-8954-df4b1aa6e6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:03.874952 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1393/2771] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:39:03.881126 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a37de1-d700-46ae-a818-63a07866c66e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:03.898986 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a37de1-d700-46ae-a818-63a07866c66e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:03.903967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d619546-1452-4bbd-ae30-a094eebf9639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:03.904648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619546-1452-4bbd-ae30-a094eebf9639 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:03.904930 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619546-1452-4bbd-ae30-a094eebf9639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:03.905208 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619546-1452-4bbd-ae30-a094eebf9639 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:03.905955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d619546-1452-4bbd-ae30-a094eebf9639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:39:03.906271 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d619546-1452-4bbd-ae30-a094eebf9639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:03.908718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d619546-1452-4bbd-ae30-a094eebf9639 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:03.926207 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1379/2772] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:39:03.931628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798aecc1-8aa9-4610-8954-df4b1aa6e6ff None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:03.943012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798aecc1-8aa9-4610-8954-df4b1aa6e6ff None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:03.947860 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a8993-285c-4b66-b693-581ece479f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:03.948524 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a8993-285c-4b66-b693-581ece479f7c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:03.948840 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a8993-285c-4b66-b693-581ece479f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:03.949122 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a8993-285c-4b66-b693-581ece479f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:03.950152 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a8993-285c-4b66-b693-581ece479f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:03.951978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a8993-285c-4b66-b693-581ece479f7c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:03.981744 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1394/2773] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:39:03 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:39:03.985790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d619546-1452-4bbd-ae30-a094eebf9639 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:03.997204 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d619546-1452-4bbd-ae30-a094eebf9639 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:04.069177 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c665f-fea7-42d8-8447-76f47972f096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:04.070115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c665f-fea7-42d8-8447-76f47972f096 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:04.070525 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c665f-fea7-42d8-8447-76f47972f096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:04.070887 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c665f-fea7-42d8-8447-76f47972f096 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:04.071703 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c665f-fea7-42d8-8447-76f47972f096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:04.073828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c665f-fea7-42d8-8447-76f47972f096 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:04.096385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1380/2774] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:39:03 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:39:04.100226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a8993-285c-4b66-b693-581ece479f7c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:04.113319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a8993-285c-4b66-b693-581ece479f7c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:04.118234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfbc65-5b48-4fc5-ba84-869c59ef1639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:04.118908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfbc65-5b48-4fc5-ba84-869c59ef1639 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:04.119198 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfbc65-5b48-4fc5-ba84-869c59ef1639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:04.119551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfbc65-5b48-4fc5-ba84-869c59ef1639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:04.120293 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfbc65-5b48-4fc5-ba84-869c59ef1639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:04.122044 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfbc65-5b48-4fc5-ba84-869c59ef1639 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:04.135974 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1395/2775] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:39:04.139997 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c665f-fea7-42d8-8447-76f47972f096 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:04.151514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c665f-fea7-42d8-8447-76f47972f096 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:04.157088 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c43d0-e6c9-4d7c-b395-689cbab6b425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:04.157835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c43d0-e6c9-4d7c-b395-689cbab6b425 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:04.158232 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c43d0-e6c9-4d7c-b395-689cbab6b425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:04.158579 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c43d0-e6c9-4d7c-b395-689cbab6b425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:04.159377 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c43d0-e6c9-4d7c-b395-689cbab6b425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:04.160996 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c43d0-e6c9-4d7c-b395-689cbab6b425 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:04.175293 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1381/2776] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:39:04.179655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfbc65-5b48-4fc5-ba84-869c59ef1639 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:04.196242 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfbc65-5b48-4fc5-ba84-869c59ef1639 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:04.205190 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0dca4-987e-44e6-926f-4e673eece95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:04.206256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0dca4-987e-44e6-926f-4e673eece95f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:04.206602 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0dca4-987e-44e6-926f-4e673eece95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:04.206954 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0dca4-987e-44e6-926f-4e673eece95f admin admin] PATH_INFO: `/v3/projects/95b0fff973e5444eae6e5dbd00dabf3b/users/60e9e4b2095e411e97024c5875d3b247/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:04.224459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0dca4-987e-44e6-926f-4e673eece95f admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b0fff973e5444eae6e5dbd00dabf3b, user_id=60e9e4b2095e411e97024c5875d3b247, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:04.227289 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0dca4-987e-44e6-926f-4e673eece95f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:04.247143 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1396/2777] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:39:04 2022] PUT /identity/v3/projects/95b0fff973e5444eae6e5dbd00dabf3b/users/60e9e4b2095e411e97024c5875d3b247/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:04.260247 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac826b-9898-4dbe-ad46-ef9ad6319563 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:04.260558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac826b-9898-4dbe-ad46-ef9ad6319563 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:04.260800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac826b-9898-4dbe-ad46-ef9ad6319563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:04.291211 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-14ac826b-9898-4dbe-ad46-ef9ad6319563 None None] MFA Rules not processed for user `60e9e4b2095e411e97024c5875d3b247`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:04.292426 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-14ac826b-9898-4dbe-ad46-ef9ad6319563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:04.377020 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1382/2778] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:04.384859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0dca4-987e-44e6-926f-4e673eece95f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:04.387029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-25c0dca4-987e-44e6-926f-4e673eece95f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:04.440368 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:04.454868 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0dca4-987e-44e6-926f-4e673eece95f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:04.459499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4eeb6f-9eea-499c-88f4-bbfe69a503d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:04.460412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4eeb6f-9eea-499c-88f4-bbfe69a503d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:04.460701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4eeb6f-9eea-499c-88f4-bbfe69a503d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:04.460983 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4eeb6f-9eea-499c-88f4-bbfe69a503d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:04.471203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4eeb6f-9eea-499c-88f4-bbfe69a503d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:04.472653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4eeb6f-9eea-499c-88f4-bbfe69a503d0 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:04.485435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1397/2779] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:04.508877 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac826b-9898-4dbe-ad46-ef9ad6319563 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:04.520899 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:04.605309 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac826b-9898-4dbe-ad46-ef9ad6319563 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:04.611588 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:04.612477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:04.612759 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:04.613036 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] PATH_INFO: `/v3/users/60e9e4b2095e411e97024c5875d3b247` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:04.614296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e9e4b2095e411e97024c5875d3b247)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:04.616229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:04.681107 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e9e4b2095e411e97024c5875d3b247'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:04.681864 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e9e4b2095e411e97024c5875d3b247'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:04.692351 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e9e4b2095e411e97024c5875d3b247'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:04.706985 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e9e4b2095e411e97024c5875d3b247'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:04.707677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e9e4b2095e411e97024c5875d3b247'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:04.728662 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1383/2780] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:39:04 2022] DELETE /identity/v3/users/60e9e4b2095e411e97024c5875d3b247 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:04.733361 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506011f7-ab4a-4bf6-a4b0-bddd71ca0f15 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:04.733873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506011f7-ab4a-4bf6-a4b0-bddd71ca0f15 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:04.734024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506011f7-ab4a-4bf6-a4b0-bddd71ca0f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:04.755453 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-506011f7-ab4a-4bf6-a4b0-bddd71ca0f15 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:04.756651 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-506011f7-ab4a-4bf6-a4b0-bddd71ca0f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:04.838211 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1398/2781] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:04.846043 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:04.847904 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:04.921117 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9742b4b-a25d-4acf-95e2-fdcd62e35987 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:04.926851 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13ab47-3bfa-49a0-99c6-15cfd673c160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:04.927541 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13ab47-3bfa-49a0-99c6-15cfd673c160 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:04.927823 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13ab47-3bfa-49a0-99c6-15cfd673c160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:04.928191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13ab47-3bfa-49a0-99c6-15cfd673c160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:04.941528 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13ab47-3bfa-49a0-99c6-15cfd673c160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:04.943078 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13ab47-3bfa-49a0-99c6-15cfd673c160 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:04.958215 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1384/2782] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:39:05.270485 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506011f7-ab4a-4bf6-a4b0-bddd71ca0f15 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:05.272093 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-506011f7-ab4a-4bf6-a4b0-bddd71ca0f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:05.292885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506011f7-ab4a-4bf6-a4b0-bddd71ca0f15 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:05.301145 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:05.302073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:05.302360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:05.302731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] PATH_INFO: `/v3/projects/95b0fff973e5444eae6e5dbd00dabf3b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:05.303998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] RBAC: Authorizing `identity:delete_project(project_id=95b0fff973e5444eae6e5dbd00dabf3b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:05.306118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:05.370628 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95b0fff973e5444eae6e5dbd00dabf3b', 'request_id': 'req-0f963c50-3be2-470f-8c6e-40036f4f353a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:05.380637 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95b0fff973e5444eae6e5dbd00dabf3b', 'request_id': 'req-0f963c50-3be2-470f-8c6e-40036f4f353a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:05.388003 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95b0fff973e5444eae6e5dbd00dabf3b', 'request_id': 'req-0f963c50-3be2-470f-8c6e-40036f4f353a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:05.399996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] The token cache is being invalidate because project 95b0fff973e5444eae6e5dbd00dabf3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:39:05.400358 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:05.401100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0f963c50-3be2-470f-8c6e-40036f4f353a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:05.402764 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1399/2783] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:39:05 2022] DELETE /identity/v3/projects/95b0fff973e5444eae6e5dbd00dabf3b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:05.410296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70cdb9-b41f-4318-bc33-c11256fe4b73 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:05.410832 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70cdb9-b41f-4318-bc33-c11256fe4b73 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:05.411072 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70cdb9-b41f-4318-bc33-c11256fe4b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:05.430922 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8c70cdb9-b41f-4318-bc33-c11256fe4b73 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:05.432299 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8c70cdb9-b41f-4318-bc33-c11256fe4b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:05.464025 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:05.506696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1385/2784] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:39:05.514301 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551f715-9d1b-455e-85eb-e9ccd0418c7b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:05.514530 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551f715-9d1b-455e-85eb-e9ccd0418c7b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:05.514841 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551f715-9d1b-455e-85eb-e9ccd0418c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:05.539449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2551f715-9d1b-455e-85eb-e9ccd0418c7b None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:05.540774 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2551f715-9d1b-455e-85eb-e9ccd0418c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:05.554190 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70cdb9-b41f-4318-bc33-c11256fe4b73 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:05.555418 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1400/2785] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:05.565634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70cdb9-b41f-4318-bc33-c11256fe4b73 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:05.571913 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfc50f-fbf6-4b09-9df6-4de7a9277524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:05.572670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfc50f-fbf6-4b09-9df6-4de7a9277524 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:05.573015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfc50f-fbf6-4b09-9df6-4de7a9277524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:05.573389 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfc50f-fbf6-4b09-9df6-4de7a9277524 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:05.574226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfc50f-fbf6-4b09-9df6-4de7a9277524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:39:05.574619 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfc50f-fbf6-4b09-9df6-4de7a9277524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:05.576321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfc50f-fbf6-4b09-9df6-4de7a9277524 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:05.589981 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1386/2786] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:39:05.593975 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551f715-9d1b-455e-85eb-e9ccd0418c7b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:05.606448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551f715-9d1b-455e-85eb-e9ccd0418c7b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:05.612177 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4b508-ba1d-4f25-9f6b-f5fe2fa1ac7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:05.612855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4b508-ba1d-4f25-9f6b-f5fe2fa1ac7b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:05.613145 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4b508-ba1d-4f25-9f6b-f5fe2fa1ac7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:05.613431 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4b508-ba1d-4f25-9f6b-f5fe2fa1ac7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:05.614164 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4b508-ba1d-4f25-9f6b-f5fe2fa1ac7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:05.616080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4b508-ba1d-4f25-9f6b-f5fe2fa1ac7b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:05.645335 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1401/2787] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:39:05 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:05.649375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfc50f-fbf6-4b09-9df6-4de7a9277524 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:05.665061 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfc50f-fbf6-4b09-9df6-4de7a9277524 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:05.735890 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff834045-9a3f-4b56-b99c-a428aecbb101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:05.736560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff834045-9a3f-4b56-b99c-a428aecbb101 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:05.736850 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff834045-9a3f-4b56-b99c-a428aecbb101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:05.737125 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff834045-9a3f-4b56-b99c-a428aecbb101 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:05.737828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff834045-9a3f-4b56-b99c-a428aecbb101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:05.739956 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff834045-9a3f-4b56-b99c-a428aecbb101 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:05.771511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1387/2788] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:39:05 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:05.772924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4b508-ba1d-4f25-9f6b-f5fe2fa1ac7b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:05.787722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4b508-ba1d-4f25-9f6b-f5fe2fa1ac7b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:05.792894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd5287-a61d-40d1-afad-83d2ed165d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:05.794394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd5287-a61d-40d1-afad-83d2ed165d46 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:05.794394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd5287-a61d-40d1-afad-83d2ed165d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:05.795005 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd5287-a61d-40d1-afad-83d2ed165d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:05.795455 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd5287-a61d-40d1-afad-83d2ed165d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:05.797126 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd5287-a61d-40d1-afad-83d2ed165d46 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:05.810439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1402/2789] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:39:05.814742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff834045-9a3f-4b56-b99c-a428aecbb101 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:05.827666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff834045-9a3f-4b56-b99c-a428aecbb101 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:05.834536 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf715ab-8a9d-4666-8956-f1025a5d794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:05.835329 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf715ab-8a9d-4666-8956-f1025a5d794d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:05.835716 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf715ab-8a9d-4666-8956-f1025a5d794d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:05.836090 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf715ab-8a9d-4666-8956-f1025a5d794d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:05.836963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf715ab-8a9d-4666-8956-f1025a5d794d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:05.838714 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf715ab-8a9d-4666-8956-f1025a5d794d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:05.861848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd5287-a61d-40d1-afad-83d2ed165d46 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:05.868282 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1388/2790] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:39:05.873195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd5287-a61d-40d1-afad-83d2ed165d46 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:05.879461 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76371796-f8dd-4bb0-8e70-60a05c30bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:05.880379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76371796-f8dd-4bb0-8e70-60a05c30bb9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:05.880666 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76371796-f8dd-4bb0-8e70-60a05c30bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:05.880911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76371796-f8dd-4bb0-8e70-60a05c30bb9e admin admin] PATH_INFO: `/v3/projects/0189eaff2c5e458f9f807caee0fad677/users/dc7b5c5f84744d5eaad9660864139b2c/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:05.909529 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76371796-f8dd-4bb0-8e70-60a05c30bb9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0189eaff2c5e458f9f807caee0fad677, user_id=dc7b5c5f84744d5eaad9660864139b2c, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:05.912352 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76371796-f8dd-4bb0-8e70-60a05c30bb9e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:05.931293 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1403/2791] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:39:05 2022] PUT /identity/v3/projects/0189eaff2c5e458f9f807caee0fad677/users/dc7b5c5f84744d5eaad9660864139b2c/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:05.941829 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2db3e-6f82-4002-bb9b-d57a35cb7d2b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:05.942222 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2db3e-6f82-4002-bb9b-d57a35cb7d2b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:05.942578 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2db3e-6f82-4002-bb9b-d57a35cb7d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:05.973381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-94b2db3e-6f82-4002-bb9b-d57a35cb7d2b None None] MFA Rules not processed for user `dc7b5c5f84744d5eaad9660864139b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:05.974753 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-94b2db3e-6f82-4002-bb9b-d57a35cb7d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:06.054201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1389/2792] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:39:06.058999 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76371796-f8dd-4bb0-8e70-60a05c30bb9e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:06.070219 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:06.135569 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76371796-f8dd-4bb0-8e70-60a05c30bb9e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:06.145010 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7852819-cb8a-4091-8883-81f221499824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:06.146059 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7852819-cb8a-4091-8883-81f221499824 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:06.146578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7852819-cb8a-4091-8883-81f221499824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:06.147080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7852819-cb8a-4091-8883-81f221499824 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:06.147854 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7852819-cb8a-4091-8883-81f221499824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:06.149883 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7852819-cb8a-4091-8883-81f221499824 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:06.185170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1404/2793] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:39:06 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:06.189230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2db3e-6f82-4002-bb9b-d57a35cb7d2b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:06.202592 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2db3e-6f82-4002-bb9b-d57a35cb7d2b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:06.229974 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:06.278652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fb886-4b0b-4ca7-ba47-13bdfda9d39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:06.279622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fb886-4b0b-4ca7-ba47-13bdfda9d39c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:06.279905 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fb886-4b0b-4ca7-ba47-13bdfda9d39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:06.280183 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fb886-4b0b-4ca7-ba47-13bdfda9d39c admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:06.280891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589fb886-4b0b-4ca7-ba47-13bdfda9d39c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:06.283901 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589fb886-4b0b-4ca7-ba47-13bdfda9d39c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:06.312569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1390/2794] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:39:06 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:39:06.317467 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7852819-cb8a-4091-8883-81f221499824 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:06.329330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7852819-cb8a-4091-8883-81f221499824 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:06.335408 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad8418-90d6-456f-8c57-afe604ae5600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:06.336517 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad8418-90d6-456f-8c57-afe604ae5600 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:06.336832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad8418-90d6-456f-8c57-afe604ae5600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:06.337121 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad8418-90d6-456f-8c57-afe604ae5600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:06.337894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad8418-90d6-456f-8c57-afe604ae5600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:06.339653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad8418-90d6-456f-8c57-afe604ae5600 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:06.360531 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1405/2795] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:39:06.362050 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fb886-4b0b-4ca7-ba47-13bdfda9d39c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:06.374696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fb886-4b0b-4ca7-ba47-13bdfda9d39c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:06.380381 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f0a40-f5c0-4f62-b2dd-8fef143a87fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:06.381054 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f0a40-f5c0-4f62-b2dd-8fef143a87fa admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:06.381342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f0a40-f5c0-4f62-b2dd-8fef143a87fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:06.381623 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f0a40-f5c0-4f62-b2dd-8fef143a87fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:06.382406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f0a40-f5c0-4f62-b2dd-8fef143a87fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:06.384015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f0a40-f5c0-4f62-b2dd-8fef143a87fa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:06.412707 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1391/2796] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:39:06.428318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad8418-90d6-456f-8c57-afe604ae5600 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:06.428318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad8418-90d6-456f-8c57-afe604ae5600 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:06.429068 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c90f83-3964-4424-b1fe-0bfb885e5298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:06.431257 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c90f83-3964-4424-b1fe-0bfb885e5298 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:06.431257 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c90f83-3964-4424-b1fe-0bfb885e5298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:06.431655 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c90f83-3964-4424-b1fe-0bfb885e5298 admin admin] PATH_INFO: `/v3/projects/a061f988650542eb85b6dc66d46b1ab8/users/e2f88891f27e45e681207c16d788cbad/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:06.447912 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c90f83-3964-4424-b1fe-0bfb885e5298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a061f988650542eb85b6dc66d46b1ab8, user_id=e2f88891f27e45e681207c16d788cbad, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:06.450648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c90f83-3964-4424-b1fe-0bfb885e5298 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:06.465487 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1406/2797] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:39:06 2022] PUT /identity/v3/projects/a061f988650542eb85b6dc66d46b1ab8/users/e2f88891f27e45e681207c16d788cbad/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:06.477671 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b2d66-8fcb-4529-a94b-c7b62d411924 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:06.478005 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b2d66-8fcb-4529-a94b-c7b62d411924 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:06.478297 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b2d66-8fcb-4529-a94b-c7b62d411924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:06.511691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-769b2d66-8fcb-4529-a94b-c7b62d411924 None None] MFA Rules not processed for user `e2f88891f27e45e681207c16d788cbad`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:06.513046 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-769b2d66-8fcb-4529-a94b-c7b62d411924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:06.590862 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1392/2798] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:06.596262 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c90f83-3964-4424-b1fe-0bfb885e5298 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:06.662538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c90f83-3964-4424-b1fe-0bfb885e5298 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:06.667254 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8e84c-6840-451b-bb36-999f811ce532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:06.667929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8e84c-6840-451b-bb36-999f811ce532 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:06.668213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8e84c-6840-451b-bb36-999f811ce532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:06.668498 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8e84c-6840-451b-bb36-999f811ce532 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:06.669195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8e84c-6840-451b-bb36-999f811ce532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:06.671170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8e84c-6840-451b-bb36-999f811ce532 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:06.702424 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1407/2799] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:39:06 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:06.707754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b2d66-8fcb-4529-a94b-c7b62d411924 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:06.719728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b2d66-8fcb-4529-a94b-c7b62d411924 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:06.784413 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3d738-5ad8-482e-8970-0f2122da1f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:06.785253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3d738-5ad8-482e-8970-0f2122da1f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:06.785639 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3d738-5ad8-482e-8970-0f2122da1f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:06.786103 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3d738-5ad8-482e-8970-0f2122da1f41 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:06.786906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3d738-5ad8-482e-8970-0f2122da1f41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:06.789008 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3d738-5ad8-482e-8970-0f2122da1f41 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:06.811174 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1393/2800] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:39:06 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:06.814904 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8e84c-6840-451b-bb36-999f811ce532 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:06.825908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8e84c-6840-451b-bb36-999f811ce532 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:06.831692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f4fe4-2fad-424f-9c85-9eec9b7c1412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:06.832580 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f4fe4-2fad-424f-9c85-9eec9b7c1412 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:06.832944 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f4fe4-2fad-424f-9c85-9eec9b7c1412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:06.833324 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f4fe4-2fad-424f-9c85-9eec9b7c1412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:06.834210 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f4fe4-2fad-424f-9c85-9eec9b7c1412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:06.835898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f4fe4-2fad-424f-9c85-9eec9b7c1412 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:06.848979 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1408/2801] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:39:06.857150 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3d738-5ad8-482e-8970-0f2122da1f41 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:06.868512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3d738-5ad8-482e-8970-0f2122da1f41 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:06.874260 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc99bf5-0d01-49d0-a86d-f779b5ec7832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:06.875735 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc99bf5-0d01-49d0-a86d-f779b5ec7832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:06.878547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc99bf5-0d01-49d0-a86d-f779b5ec7832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:06.884729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc99bf5-0d01-49d0-a86d-f779b5ec7832 admin admin] PATH_INFO: `/v3/projects/be85b9c8011543dcb07b923e2320ce0a/users/60a4e5c735c5464c9814ea817d54c4f0/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:06.902438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc99bf5-0d01-49d0-a86d-f779b5ec7832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be85b9c8011543dcb07b923e2320ce0a, user_id=60a4e5c735c5464c9814ea817d54c4f0, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:06.905561 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc99bf5-0d01-49d0-a86d-f779b5ec7832 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:06.915863 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:06.927677 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1394/2802] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:39:06 2022] PUT /identity/v3/projects/be85b9c8011543dcb07b923e2320ce0a/users/60a4e5c735c5464c9814ea817d54c4f0/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:06.932027 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f4fe4-2fad-424f-9c85-9eec9b7c1412 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:06.942598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f4fe4-2fad-424f-9c85-9eec9b7c1412 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:06.974318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:07.008094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a33677-8193-46c0-ae31-5c1d503f394c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:07.008832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a33677-8193-46c0-ae31-5c1d503f394c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:07.009171 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a33677-8193-46c0-ae31-5c1d503f394c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:07.009509 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a33677-8193-46c0-ae31-5c1d503f394c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:07.010437 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a33677-8193-46c0-ae31-5c1d503f394c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:07.012234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a33677-8193-46c0-ae31-5c1d503f394c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:07.023043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1409/2803] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:39:07.027234 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc99bf5-0d01-49d0-a86d-f779b5ec7832 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:07.039143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc99bf5-0d01-49d0-a86d-f779b5ec7832 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:07.044347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfccef-3611-4b98-b63f-3e5cdd940f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:07.045242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfccef-3611-4b98-b63f-3e5cdd940f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:07.045522 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfccef-3611-4b98-b63f-3e5cdd940f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:07.045816 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfccef-3611-4b98-b63f-3e5cdd940f67 admin admin] PATH_INFO: `/v3/projects/be85b9c8011543dcb07b923e2320ce0a/users/60a4e5c735c5464c9814ea817d54c4f0/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:07.048341 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfccef-3611-4b98-b63f-3e5cdd940f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be85b9c8011543dcb07b923e2320ce0a, user_id=60a4e5c735c5464c9814ea817d54c4f0, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:07.051691 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfccef-3611-4b98-b63f-3e5cdd940f67 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:07.063498 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6433210-a118-4050-8d57-e0c231a3c7f8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:07.063708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1395/2804] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:39:07 2022] PUT /identity/v3/projects/be85b9c8011543dcb07b923e2320ce0a/users/60a4e5c735c5464c9814ea817d54c4f0/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:07.063839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6433210-a118-4050-8d57-e0c231a3c7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:07.064045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6433210-a118-4050-8d57-e0c231a3c7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:07.073980 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98585d42-c3d3-4db2-aa33-9db156030462 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:07.074408 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98585d42-c3d3-4db2-aa33-9db156030462 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:07.074789 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98585d42-c3d3-4db2-aa33-9db156030462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:07.082536 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c6433210-a118-4050-8d57-e0c231a3c7f8 None None] MFA Rules not processed for user `756c74602c4d4bafade01c406ffd6afc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:07.083752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c6433210-a118-4050-8d57-e0c231a3c7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:07.104742 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-98585d42-c3d3-4db2-aa33-9db156030462 None None] MFA Rules not processed for user `60a4e5c735c5464c9814ea817d54c4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:07.105857 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-98585d42-c3d3-4db2-aa33-9db156030462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:07.139898 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1410/2805] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:39:07.177490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1396/2806] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:39:07.189627 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6433210-a118-4050-8d57-e0c231a3c7f8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:07.191253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c6433210-a118-4050-8d57-e0c231a3c7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:07.249141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6433210-a118-4050-8d57-e0c231a3c7f8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:07.253360 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e155a2b-d813-43d8-a547-744709a60e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:07.254173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e155a2b-d813-43d8-a547-744709a60e30 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:07.254460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e155a2b-d813-43d8-a547-744709a60e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:07.254875 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e155a2b-d813-43d8-a547-744709a60e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:07.314473 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e155a2b-d813-43d8-a547-744709a60e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:07.315935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e155a2b-d813-43d8-a547-744709a60e30 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:07.325499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1411/2807] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:08.185720 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98585d42-c3d3-4db2-aa33-9db156030462 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:08.187375 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-98585d42-c3d3-4db2-aa33-9db156030462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:08.277915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98585d42-c3d3-4db2-aa33-9db156030462 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:08.283806 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5379be6-1dbb-429b-a37c-1dea9a946f86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da2d85eb18049f196aeefc51d3f1e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:08.284479 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5379be6-1dbb-429b-a37c-1dea9a946f86 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:08.284868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5379be6-1dbb-429b-a37c-1dea9a946f86 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:08.285147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5379be6-1dbb-429b-a37c-1dea9a946f86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:08.286967 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d5379be6-1dbb-429b-a37c-1dea9a946f86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:08.299428 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5379be6-1dbb-429b-a37c-1dea9a946f86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:08.301077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5379be6-1dbb-429b-a37c-1dea9a946f86 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:08.311056 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1397/2808] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:08.827001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e155a2b-d813-43d8-a547-744709a60e30 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:08.829220 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7e155a2b-d813-43d8-a547-744709a60e30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:08.841052 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e155a2b-d813-43d8-a547-744709a60e30 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:08.844863 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ff2c5-f91c-4e6a-9b81-ba49c8e0a935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:08.845566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ff2c5-f91c-4e6a-9b81-ba49c8e0a935 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:08.845929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ff2c5-f91c-4e6a-9b81-ba49c8e0a935 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:08.846247 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ff2c5-f91c-4e6a-9b81-ba49c8e0a935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:08.872613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:08.906433 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ff2c5-f91c-4e6a-9b81-ba49c8e0a935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:08.907909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ff2c5-f91c-4e6a-9b81-ba49c8e0a935 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:08.917249 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1412/2809] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:09.283757 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294ab16-f880-4bce-9778-79d99575e667 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:09.284313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294ab16-f880-4bce-9778-79d99575e667 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:09.284313 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294ab16-f880-4bce-9778-79d99575e667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:09.304191 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5294ab16-f880-4bce-9778-79d99575e667 None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:09.305431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5294ab16-f880-4bce-9778-79d99575e667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:09.350077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:09.382490 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1398/2810] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 06:39:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:09.398708 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ff2c5-f91c-4e6a-9b81-ba49c8e0a935 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:09.400261 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-666ff2c5-f91c-4e6a-9b81-ba49c8e0a935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:09.484561 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ff2c5-f91c-4e6a-9b81-ba49c8e0a935 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:09.496153 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255edc5a-80c4-4952-af6f-397a7ee0e7d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:09.496828 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255edc5a-80c4-4952-af6f-397a7ee0e7d8 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:09.497357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255edc5a-80c4-4952-af6f-397a7ee0e7d8 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:09.497357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255edc5a-80c4-4952-af6f-397a7ee0e7d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:09.507855 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255edc5a-80c4-4952-af6f-397a7ee0e7d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:09.509670 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255edc5a-80c4-4952-af6f-397a7ee0e7d8 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:09.522124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1413/2811] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:39:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:39:09.535794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294ab16-f880-4bce-9778-79d99575e667 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:09.546794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294ab16-f880-4bce-9778-79d99575e667 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:09.551834 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe324-ee76-471d-a015-d7890e0927cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:09.552306 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe324-ee76-471d-a015-d7890e0927cb service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:09.552590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe324-ee76-471d-a015-d7890e0927cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:09.552868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe324-ee76-471d-a015-d7890e0927cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:09.562836 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfe324-ee76-471d-a015-d7890e0927cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:09.564483 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfe324-ee76-471d-a015-d7890e0927cb service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:09.591507 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1399/2812] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:13.594942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255edc5a-80c4-4952-af6f-397a7ee0e7d8 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:13.696614 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255edc5a-80c4-4952-af6f-397a7ee0e7d8 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:13.701653 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:13.702678 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:13.703033 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:13.703379 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] PATH_INFO: `/v3/users/dc7b5c5f84744d5eaad9660864139b2c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:13.704923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc7b5c5f84744d5eaad9660864139b2c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:13.706984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:13.818533 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc7b5c5f84744d5eaad9660864139b2c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:13.829825 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:13.844305 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc7b5c5f84744d5eaad9660864139b2c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:13.852984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc7b5c5f84744d5eaad9660864139b2c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:13.853786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc7b5c5f84744d5eaad9660864139b2c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:13.854448 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc7b5c5f84744d5eaad9660864139b2c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:13.872716 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1414/2813] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:39:13 2022] DELETE /identity/v3/users/dc7b5c5f84744d5eaad9660864139b2c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:13.881075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226070c-91e8-419b-a40f-261c77d35d34 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:13.882026 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226070c-91e8-419b-a40f-261c77d35d34 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:13.882455 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226070c-91e8-419b-a40f-261c77d35d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:13.903564 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a226070c-91e8-419b-a40f-261c77d35d34 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:13.905201 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a226070c-91e8-419b-a40f-261c77d35d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:13.983092 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1400/2814] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:13.996406 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:13.997888 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:14.075911 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a697b5-733d-421a-b395-06b842204dd5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:14.081913 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15898b51-cc20-4f7c-8905-264a67f93450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:14.083058 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15898b51-cc20-4f7c-8905-264a67f93450 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:14.083537 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15898b51-cc20-4f7c-8905-264a67f93450 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:14.083859 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15898b51-cc20-4f7c-8905-264a67f93450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:14.094114 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15898b51-cc20-4f7c-8905-264a67f93450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:14.096245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15898b51-cc20-4f7c-8905-264a67f93450 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:14.107162 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1415/2815] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:14.468267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226070c-91e8-419b-a40f-261c77d35d34 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:14.470795 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a226070c-91e8-419b-a40f-261c77d35d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:14.493242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226070c-91e8-419b-a40f-261c77d35d34 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:14.502871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:14.504335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:14.504445 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:14.504846 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] PATH_INFO: `/v3/projects/0189eaff2c5e458f9f807caee0fad677` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:14.506477 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0189eaff2c5e458f9f807caee0fad677)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:14.508650 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:14.569648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0189eaff2c5e458f9f807caee0fad677', 'request_id': 'req-b3241ae1-a89f-42c7-b29c-70a13c491177'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:14.577357 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0189eaff2c5e458f9f807caee0fad677', 'request_id': 'req-b3241ae1-a89f-42c7-b29c-70a13c491177'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:14.586044 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0189eaff2c5e458f9f807caee0fad677', 'request_id': 'req-b3241ae1-a89f-42c7-b29c-70a13c491177'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:14.604009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] The token cache is being invalidate because project 0189eaff2c5e458f9f807caee0fad677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:39:14.604760 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:14.605777 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:14.611052 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1401/2816] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:39:14 2022] DELETE /identity/v3/projects/0189eaff2c5e458f9f807caee0fad677 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:14.612503 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15898b51-cc20-4f7c-8905-264a67f93450 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:14.622513 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15898b51-cc20-4f7c-8905-264a67f93450 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:14.682890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:14.683785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:14.684071 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:14.684357 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] PATH_INFO: `/v3/users/e2f88891f27e45e681207c16d788cbad` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:14.685968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2f88891f27e45e681207c16d788cbad)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:14.689029 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:14.745929 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2f88891f27e45e681207c16d788cbad'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:14.760832 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2f88891f27e45e681207c16d788cbad'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:14.777454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2f88891f27e45e681207c16d788cbad'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:14.780170 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2f88891f27e45e681207c16d788cbad'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:14.782996 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2f88891f27e45e681207c16d788cbad'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:14.787011 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:14.817203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1416/2817] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:39:14 2022] DELETE /identity/v3/users/e2f88891f27e45e681207c16d788cbad => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:15.161593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:15.163935 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:15.186540 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3241ae1-a89f-42c7-b29c-70a13c491177 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:15.251378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:15.259803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:15.260675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:15.261446 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:15.261895 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] PATH_INFO: `/v3/projects/a061f988650542eb85b6dc66d46b1ab8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:15.263473 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a061f988650542eb85b6dc66d46b1ab8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:15.265843 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:15.318560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a061f988650542eb85b6dc66d46b1ab8', 'request_id': 'req-9d76f24f-e542-4d1b-8be5-4b784006a004'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:15.324793 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a061f988650542eb85b6dc66d46b1ab8', 'request_id': 'req-9d76f24f-e542-4d1b-8be5-4b784006a004'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:15.335601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a061f988650542eb85b6dc66d46b1ab8', 'request_id': 'req-9d76f24f-e542-4d1b-8be5-4b784006a004'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:15.349828 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] The token cache is being invalidate because project a061f988650542eb85b6dc66d46b1ab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:39:15.350099 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:15.350696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:15.354018 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1402/2818] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:39:15 2022] DELETE /identity/v3/projects/a061f988650542eb85b6dc66d46b1ab8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:15.356287 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:15.357699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:15.371751 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1a7ad-35f3-4769-938e-33c88c147822 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:15.443454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c80df8-4524-4298-96fe-b8abe789a0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:15.444288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c80df8-4524-4298-96fe-b8abe789a0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:15.444571 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c80df8-4524-4298-96fe-b8abe789a0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:15.444845 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c80df8-4524-4298-96fe-b8abe789a0a2 admin admin] PATH_INFO: `/v3/users/60a4e5c735c5464c9814ea817d54c4f0` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:15.446083 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c80df8-4524-4298-96fe-b8abe789a0a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a4e5c735c5464c9814ea817d54c4f0)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:15.448082 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c80df8-4524-4298-96fe-b8abe789a0a2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:15.504479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73c80df8-4524-4298-96fe-b8abe789a0a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a4e5c735c5464c9814ea817d54c4f0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:15.520951 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73c80df8-4524-4298-96fe-b8abe789a0a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a4e5c735c5464c9814ea817d54c4f0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:15.527878 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73c80df8-4524-4298-96fe-b8abe789a0a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a4e5c735c5464c9814ea817d54c4f0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:15.528617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73c80df8-4524-4298-96fe-b8abe789a0a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a4e5c735c5464c9814ea817d54c4f0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:15.529496 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73c80df8-4524-4298-96fe-b8abe789a0a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a4e5c735c5464c9814ea817d54c4f0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:15.548252 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1417/2819] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:39:15 2022] DELETE /identity/v3/users/60a4e5c735c5464c9814ea817d54c4f0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:15.861670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:15.863511 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:15.888976 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76f24f-e542-4d1b-8be5-4b784006a004 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:15.959380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:15.960186 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:15.960471 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:15.960748 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] PATH_INFO: `/v3/projects/be85b9c8011543dcb07b923e2320ce0a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:15.962111 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=be85b9c8011543dcb07b923e2320ce0a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:15.964242 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:16.019261 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be85b9c8011543dcb07b923e2320ce0a', 'request_id': 'req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:16.025059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be85b9c8011543dcb07b923e2320ce0a', 'request_id': 'req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:16.032909 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be85b9c8011543dcb07b923e2320ce0a', 'request_id': 'req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:16.046615 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] The token cache is being invalidate because project be85b9c8011543dcb07b923e2320ce0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:39:16.047180 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:16.058476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51e88480-bcd2-42eb-bb0b-a8ec4cfd32cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:16.060176 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1403/2820] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:39:15 2022] DELETE /identity/v3/projects/be85b9c8011543dcb07b923e2320ce0a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:16.064965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163d29f-914a-494b-a552-34f5332cc3f2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:16.065551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163d29f-914a-494b-a552-34f5332cc3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:16.065551 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163d29f-914a-494b-a552-34f5332cc3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:16.088729 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e163d29f-914a-494b-a552-34f5332cc3f2 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:16.090600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e163d29f-914a-494b-a552-34f5332cc3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:16.208292 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1418/2821] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:16.220439 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0f300-efd8-4aea-9caa-da7bd695e24c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:16.221159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0f300-efd8-4aea-9caa-da7bd695e24c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:16.221159 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0f300-efd8-4aea-9caa-da7bd695e24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:16.247442 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b4e0f300-efd8-4aea-9caa-da7bd695e24c None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:16.248861 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b4e0f300-efd8-4aea-9caa-da7bd695e24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:16.257416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1404/2822] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:16.261703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163d29f-914a-494b-a552-34f5332cc3f2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:16.273090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163d29f-914a-494b-a552-34f5332cc3f2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:16.279926 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb93ea-60d9-48c9-a48d-270f557f0bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:16.281458 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb93ea-60d9-48c9-a48d-270f557f0bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:16.281747 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb93ea-60d9-48c9-a48d-270f557f0bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:16.282692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb93ea-60d9-48c9-a48d-270f557f0bb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:16.283474 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb93ea-60d9-48c9-a48d-270f557f0bb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:39:16.283786 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb93ea-60d9-48c9-a48d-270f557f0bb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:16.285412 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb93ea-60d9-48c9-a48d-270f557f0bb6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:16.299085 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1419/2823] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:39:16.303558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0f300-efd8-4aea-9caa-da7bd695e24c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:16.315279 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0f300-efd8-4aea-9caa-da7bd695e24c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:16.320132 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a146070-32cb-4cbc-802e-486154235c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:16.320879 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a146070-32cb-4cbc-802e-486154235c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:16.321351 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a146070-32cb-4cbc-802e-486154235c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:16.321722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a146070-32cb-4cbc-802e-486154235c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:16.322530 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a146070-32cb-4cbc-802e-486154235c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:16.324534 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a146070-32cb-4cbc-802e-486154235c4d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:16.329144 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:16.353919 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1405/2824] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:39:16 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:16.359155 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb93ea-60d9-48c9-a48d-270f557f0bb6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:16.366277 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:16.373512 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb93ea-60d9-48c9-a48d-270f557f0bb6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:16.446924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2118d-5621-48e7-9133-d64616584301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:16.448076 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2118d-5621-48e7-9133-d64616584301 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:16.448609 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2118d-5621-48e7-9133-d64616584301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:16.449133 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2118d-5621-48e7-9133-d64616584301 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:16.450138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2118d-5621-48e7-9133-d64616584301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:16.452080 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2118d-5621-48e7-9133-d64616584301 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:16.476612 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1420/2825] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:39:16 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:16.477888 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a146070-32cb-4cbc-802e-486154235c4d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:16.492385 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a146070-32cb-4cbc-802e-486154235c4d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:16.497538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2f0ef-66d5-4131-b1c3-a453ad01aea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:16.498287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2f0ef-66d5-4131-b1c3-a453ad01aea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:16.498560 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2f0ef-66d5-4131-b1c3-a453ad01aea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:16.498842 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2f0ef-66d5-4131-b1c3-a453ad01aea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:16.499590 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2f0ef-66d5-4131-b1c3-a453ad01aea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:16.501353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2f0ef-66d5-4131-b1c3-a453ad01aea2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:16.515614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1406/2826] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:39:16.516693 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2118d-5621-48e7-9133-d64616584301 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:16.527435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2118d-5621-48e7-9133-d64616584301 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:16.533025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98eba80-857b-4ca5-b685-ad9a76157ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:16.533839 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eba80-857b-4ca5-b685-ad9a76157ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:16.534215 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eba80-857b-4ca5-b685-ad9a76157ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:16.534510 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eba80-857b-4ca5-b685-ad9a76157ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:16.535270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98eba80-857b-4ca5-b685-ad9a76157ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:16.536935 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98eba80-857b-4ca5-b685-ad9a76157ccc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:16.548752 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1421/2827] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:39:16.552778 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2f0ef-66d5-4131-b1c3-a453ad01aea2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:16.564074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2f0ef-66d5-4131-b1c3-a453ad01aea2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:16.570080 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710a8c6-a54a-4827-a6f8-2c217d0857ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:16.571065 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710a8c6-a54a-4827-a6f8-2c217d0857ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:16.571347 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710a8c6-a54a-4827-a6f8-2c217d0857ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:16.571637 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710a8c6-a54a-4827-a6f8-2c217d0857ac admin admin] PATH_INFO: `/v3/projects/e060665380904144be241b65fea9d2ee/users/e6ffe75bad02473bbd4a62cd9e35d434/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:16.590141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710a8c6-a54a-4827-a6f8-2c217d0857ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e060665380904144be241b65fea9d2ee, user_id=e6ffe75bad02473bbd4a62cd9e35d434, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:16.592809 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710a8c6-a54a-4827-a6f8-2c217d0857ac admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:16.608618 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1407/2828] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:39:16 2022] PUT /identity/v3/projects/e060665380904144be241b65fea9d2ee/users/e6ffe75bad02473bbd4a62cd9e35d434/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:16.676643 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195249b-ba0e-42a0-91d9-8eac3962afaf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:16.676962 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195249b-ba0e-42a0-91d9-8eac3962afaf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:16.677218 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195249b-ba0e-42a0-91d9-8eac3962afaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:16.708731 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9195249b-ba0e-42a0-91d9-8eac3962afaf None None] MFA Rules not processed for user `e6ffe75bad02473bbd4a62cd9e35d434`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:16.710230 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9195249b-ba0e-42a0-91d9-8eac3962afaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:16.785978 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1422/2829] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:16.790570 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710a8c6-a54a-4827-a6f8-2c217d0857ac admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:16.857075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710a8c6-a54a-4827-a6f8-2c217d0857ac admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:16.862100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d60c7c-fe8d-4b02-a029-6f94e7445b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:16.862784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60c7c-fe8d-4b02-a029-6f94e7445b8f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:16.863076 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60c7c-fe8d-4b02-a029-6f94e7445b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:16.863366 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60c7c-fe8d-4b02-a029-6f94e7445b8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:16.864069 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d60c7c-fe8d-4b02-a029-6f94e7445b8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:16.866055 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d60c7c-fe8d-4b02-a029-6f94e7445b8f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:16.894514 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1408/2830] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:39:16 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:16.898460 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195249b-ba0e-42a0-91d9-8eac3962afaf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:16.910801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195249b-ba0e-42a0-91d9-8eac3962afaf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:16.993836 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63334e85-c5b8-43f2-b578-089f5ef232d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:16.994216 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63334e85-c5b8-43f2-b578-089f5ef232d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:16.994697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63334e85-c5b8-43f2-b578-089f5ef232d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:16.995181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63334e85-c5b8-43f2-b578-089f5ef232d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:16.995973 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63334e85-c5b8-43f2-b578-089f5ef232d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:17.004605 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63334e85-c5b8-43f2-b578-089f5ef232d8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:17.033443 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:17.039638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1423/2831] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:39:16 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:17.043365 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d60c7c-fe8d-4b02-a029-6f94e7445b8f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:17.056868 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d60c7c-fe8d-4b02-a029-6f94e7445b8f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:17.065258 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea10286-90ec-41b5-87a6-59f779fd54fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:17.066115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea10286-90ec-41b5-87a6-59f779fd54fe admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:17.066544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea10286-90ec-41b5-87a6-59f779fd54fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:17.066891 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea10286-90ec-41b5-87a6-59f779fd54fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:17.067704 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea10286-90ec-41b5-87a6-59f779fd54fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:17.069601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea10286-90ec-41b5-87a6-59f779fd54fe admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:17.084245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1409/2832] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:39:17.088860 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63334e85-c5b8-43f2-b578-089f5ef232d8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:17.107942 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63334e85-c5b8-43f2-b578-089f5ef232d8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:17.119140 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995609a-7fb0-4513-a183-e467947e4215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:17.120168 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995609a-7fb0-4513-a183-e467947e4215 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:17.120565 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995609a-7fb0-4513-a183-e467947e4215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:17.121018 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995609a-7fb0-4513-a183-e467947e4215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:17.121873 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995609a-7fb0-4513-a183-e467947e4215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:17.123619 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995609a-7fb0-4513-a183-e467947e4215 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:17.137553 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1424/2833] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:39:17.158495 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea10286-90ec-41b5-87a6-59f779fd54fe admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:17.203312 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea10286-90ec-41b5-87a6-59f779fd54fe admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:17.209797 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79d848-2269-47c4-854e-52f8a244bd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:17.211145 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79d848-2269-47c4-854e-52f8a244bd4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:17.211431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79d848-2269-47c4-854e-52f8a244bd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:17.211781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79d848-2269-47c4-854e-52f8a244bd4c admin admin] PATH_INFO: `/v3/projects/ba8972cf9a2841c9a46c1d2e70c036c9/users/bbcf4b14b2274b7f8877f18e77506b2c/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:17.231870 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec79d848-2269-47c4-854e-52f8a244bd4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba8972cf9a2841c9a46c1d2e70c036c9, user_id=bbcf4b14b2274b7f8877f18e77506b2c, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:17.234585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec79d848-2269-47c4-854e-52f8a244bd4c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:17.258796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1410/2834] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:39:17 2022] PUT /identity/v3/projects/ba8972cf9a2841c9a46c1d2e70c036c9/users/bbcf4b14b2274b7f8877f18e77506b2c/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:17.269478 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56fea8-d2dd-46b3-ab92-753426d01e75 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:17.269908 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56fea8-d2dd-46b3-ab92-753426d01e75 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:17.270330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56fea8-d2dd-46b3-ab92-753426d01e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:17.301521 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ad56fea8-d2dd-46b3-ab92-753426d01e75 None None] MFA Rules not processed for user `bbcf4b14b2274b7f8877f18e77506b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:17.303028 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ad56fea8-d2dd-46b3-ab92-753426d01e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:17.384566 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1425/2835] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:17.392131 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79d848-2269-47c4-854e-52f8a244bd4c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:17.462057 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79d848-2269-47c4-854e-52f8a244bd4c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:17.468921 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ec2f8-e2f4-4936-8c51-dd952022b0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:17.470568 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ec2f8-e2f4-4936-8c51-dd952022b0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:17.470752 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ec2f8-e2f4-4936-8c51-dd952022b0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:17.471015 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ec2f8-e2f4-4936-8c51-dd952022b0f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:17.471736 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ec2f8-e2f4-4936-8c51-dd952022b0f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:17.474370 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ec2f8-e2f4-4936-8c51-dd952022b0f0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:17.479754 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:17.503769 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1411/2836] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:39:17 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:17.507968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56fea8-d2dd-46b3-ab92-753426d01e75 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:17.519775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56fea8-d2dd-46b3-ab92-753426d01e75 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:17.576025 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91902348-013e-426c-b0cf-9566ccfe395c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:17.576624 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91902348-013e-426c-b0cf-9566ccfe395c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:17.576909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91902348-013e-426c-b0cf-9566ccfe395c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:17.577188 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91902348-013e-426c-b0cf-9566ccfe395c admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:17.577894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91902348-013e-426c-b0cf-9566ccfe395c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:17.580387 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91902348-013e-426c-b0cf-9566ccfe395c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:17.600700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1426/2837] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:39:17 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:17.604713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ec2f8-e2f4-4936-8c51-dd952022b0f0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:17.617052 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ec2f8-e2f4-4936-8c51-dd952022b0f0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:17.621731 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41427c-6db6-4c2a-9627-e626468fc0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:17.622438 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41427c-6db6-4c2a-9627-e626468fc0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:17.622726 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41427c-6db6-4c2a-9627-e626468fc0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:17.623009 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41427c-6db6-4c2a-9627-e626468fc0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:17.623750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41427c-6db6-4c2a-9627-e626468fc0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:17.625349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41427c-6db6-4c2a-9627-e626468fc0f6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:17.636380 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1412/2838] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:39:17.640350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91902348-013e-426c-b0cf-9566ccfe395c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:17.650787 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91902348-013e-426c-b0cf-9566ccfe395c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:17.655577 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bff392-3477-4484-861f-deae808af908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:17.656449 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bff392-3477-4484-861f-deae808af908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:17.656733 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bff392-3477-4484-861f-deae808af908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:17.657827 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bff392-3477-4484-861f-deae808af908 admin admin] PATH_INFO: `/v3/projects/36bdd8c845da4b4d8ea9bbce2b877eca/users/c5430580282244dd8198c0a4a49bf878/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:17.671957 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bff392-3477-4484-861f-deae808af908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36bdd8c845da4b4d8ea9bbce2b877eca, user_id=c5430580282244dd8198c0a4a49bf878, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:17.674870 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bff392-3477-4484-861f-deae808af908 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:17.687642 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1427/2839] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:39:17 2022] PUT /identity/v3/projects/36bdd8c845da4b4d8ea9bbce2b877eca/users/c5430580282244dd8198c0a4a49bf878/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:17.692071 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41427c-6db6-4c2a-9627-e626468fc0f6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:17.702026 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41427c-6db6-4c2a-9627-e626468fc0f6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:17.757614 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f36fe4-5802-48fc-8520-5bb8b65acaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:17.758400 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f36fe4-5802-48fc-8520-5bb8b65acaec admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:17.758685 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f36fe4-5802-48fc-8520-5bb8b65acaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:17.758964 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f36fe4-5802-48fc-8520-5bb8b65acaec admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:17.759681 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f36fe4-5802-48fc-8520-5bb8b65acaec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:17.761264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f36fe4-5802-48fc-8520-5bb8b65acaec admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:17.771558 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1413/2840] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:39:17.775660 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bff392-3477-4484-861f-deae808af908 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:17.786318 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bff392-3477-4484-861f-deae808af908 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:17.790887 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ddf49-3ab8-47b0-8856-89a31203fca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:17.791767 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ddf49-3ab8-47b0-8856-89a31203fca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:17.792050 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ddf49-3ab8-47b0-8856-89a31203fca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:17.792326 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ddf49-3ab8-47b0-8856-89a31203fca8 admin admin] PATH_INFO: `/v3/projects/36bdd8c845da4b4d8ea9bbce2b877eca/users/c5430580282244dd8198c0a4a49bf878/roles/737dda9a674348dc8a8afe8f7827fa89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:17.794353 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ddf49-3ab8-47b0-8856-89a31203fca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36bdd8c845da4b4d8ea9bbce2b877eca, user_id=c5430580282244dd8198c0a4a49bf878, role_id=737dda9a674348dc8a8afe8f7827fa89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:17.796970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ddf49-3ab8-47b0-8856-89a31203fca8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:17.807349 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1428/2841] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:39:17 2022] PUT /identity/v3/projects/36bdd8c845da4b4d8ea9bbce2b877eca/users/c5430580282244dd8198c0a4a49bf878/roles/737dda9a674348dc8a8afe8f7827fa89 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:17.818969 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159b3ce-aeb0-4dda-9ff5-cd46e6ab59a0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:17.819264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159b3ce-aeb0-4dda-9ff5-cd46e6ab59a0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:17.819547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159b3ce-aeb0-4dda-9ff5-cd46e6ab59a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:17.847272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b159b3ce-aeb0-4dda-9ff5-cd46e6ab59a0 None None] MFA Rules not processed for user `c5430580282244dd8198c0a4a49bf878`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:17.848415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b159b3ce-aeb0-4dda-9ff5-cd46e6ab59a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:17.916304 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1414/2842] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:17.928354 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ddf49-3ab8-47b0-8856-89a31203fca8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:17.930498 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d78ddf49-3ab8-47b0-8856-89a31203fca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:17.966593 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:17.997959 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ddf49-3ab8-47b0-8856-89a31203fca8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:18.002438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1100e5-1d46-4f0b-b4dd-a8ba2ab15dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:18.003176 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1100e5-1d46-4f0b-b4dd-a8ba2ab15dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:18.003440 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1100e5-1d46-4f0b-b4dd-a8ba2ab15dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:18.003759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1100e5-1d46-4f0b-b4dd-a8ba2ab15dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:18.064972 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1100e5-1d46-4f0b-b4dd-a8ba2ab15dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:18.066479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1100e5-1d46-4f0b-b4dd-a8ba2ab15dc3 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:18.075378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1429/2843] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:19.600418 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159b3ce-aeb0-4dda-9ff5-cd46e6ab59a0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:19.602437 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b159b3ce-aeb0-4dda-9ff5-cd46e6ab59a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:19.618549 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159b3ce-aeb0-4dda-9ff5-cd46e6ab59a0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:19.622670 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ebc47-a14f-4a52-973c-d56ba989d6c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:19.623705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ebc47-a14f-4a52-973c-d56ba989d6c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:19.624164 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ebc47-a14f-4a52-973c-d56ba989d6c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:19.624739 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ebc47-a14f-4a52-973c-d56ba989d6c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:19.690287 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:19.694168 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ebc47-a14f-4a52-973c-d56ba989d6c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:19.695721 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ebc47-a14f-4a52-973c-d56ba989d6c7 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:19.708911 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1415/2844] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:20.294676 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1100e5-1d46-4f0b-b4dd-a8ba2ab15dc3 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:20.304330 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1100e5-1d46-4f0b-b4dd-a8ba2ab15dc3 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:20.308019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c18f81-0f83-43fe-a9f2-6e81b81bfc2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:20.308672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18f81-0f83-43fe-a9f2-6e81b81bfc2b service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:20.308960 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18f81-0f83-43fe-a9f2-6e81b81bfc2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:20.309240 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18f81-0f83-43fe-a9f2-6e81b81bfc2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:20.318462 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c18f81-0f83-43fe-a9f2-6e81b81bfc2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:20.319894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c18f81-0f83-43fe-a9f2-6e81b81bfc2b service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:20.330637 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1430/2845] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:39:24.168657 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ebc47-a14f-4a52-973c-d56ba989d6c7 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:24.233081 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ebc47-a14f-4a52-973c-d56ba989d6c7 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:24.238167 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:24.239035 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:24.239315 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:24.239591 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] PATH_INFO: `/v3/users/e6ffe75bad02473bbd4a62cd9e35d434` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:24.240768 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ffe75bad02473bbd4a62cd9e35d434)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:24.242711 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:24.290221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ffe75bad02473bbd4a62cd9e35d434'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:24.290881 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ffe75bad02473bbd4a62cd9e35d434'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:24.298697 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ffe75bad02473bbd4a62cd9e35d434'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:24.310977 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ffe75bad02473bbd4a62cd9e35d434'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:24.311601 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ffe75bad02473bbd4a62cd9e35d434'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:24.325045 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1416/2846] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:39:24 2022] DELETE /identity/v3/users/e6ffe75bad02473bbd4a62cd9e35d434 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:24.330382 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2c855-3b9f-41f3-8784-39b0ffa8f540 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:24.330899 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2c855-3b9f-41f3-8784-39b0ffa8f540 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:24.331015 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2c855-3b9f-41f3-8784-39b0ffa8f540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:24.348546 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8cb2c855-3b9f-41f3-8784-39b0ffa8f540 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:24.349724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8cb2c855-3b9f-41f3-8784-39b0ffa8f540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:24.410885 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1431/2847] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:24.420431 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:24.421810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:24.481169 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a6989-b0b2-44ae-9c6e-098b76a74162 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:24.484820 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed789891-aa7d-49e3-9de2-d9fd72905b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:24.486053 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed789891-aa7d-49e3-9de2-d9fd72905b00 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:24.486342 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed789891-aa7d-49e3-9de2-d9fd72905b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:24.486621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed789891-aa7d-49e3-9de2-d9fd72905b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:24.495675 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed789891-aa7d-49e3-9de2-d9fd72905b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:24.497096 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed789891-aa7d-49e3-9de2-d9fd72905b00 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:24.507229 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1417/2848] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:24.809617 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2c855-3b9f-41f3-8784-39b0ffa8f540 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:24.811094 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8cb2c855-3b9f-41f3-8784-39b0ffa8f540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:24.823118 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2c855-3b9f-41f3-8784-39b0ffa8f540 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:24.827876 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:24.828746 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:24.829056 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:24.829337 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] PATH_INFO: `/v3/projects/e060665380904144be241b65fea9d2ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:24.830701 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e060665380904144be241b65fea9d2ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:24.832648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:24.880265 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e060665380904144be241b65fea9d2ee', 'request_id': 'req-538f2ae2-039a-422f-bb6b-89f01bae0c68'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:24.887441 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e060665380904144be241b65fea9d2ee', 'request_id': 'req-538f2ae2-039a-422f-bb6b-89f01bae0c68'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:24.894181 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e060665380904144be241b65fea9d2ee', 'request_id': 'req-538f2ae2-039a-422f-bb6b-89f01bae0c68'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:24.898856 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:24.906200 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] The token cache is being invalidate because project e060665380904144be241b65fea9d2ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:39:24.906498 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:24.907813 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:24.909690 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1432/2849] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:39:24 2022] DELETE /identity/v3/projects/e060665380904144be241b65fea9d2ee => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:24.913796 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed789891-aa7d-49e3-9de2-d9fd72905b00 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:24.924700 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed789891-aa7d-49e3-9de2-d9fd72905b00 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:24.937230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:24.982903 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:24.983779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:24.984059 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:24.984337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] PATH_INFO: `/v3/users/bbcf4b14b2274b7f8877f18e77506b2c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:24.985523 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbcf4b14b2274b7f8877f18e77506b2c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:24.987466 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:25.032995 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbcf4b14b2274b7f8877f18e77506b2c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.033646 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbcf4b14b2274b7f8877f18e77506b2c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.040727 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbcf4b14b2274b7f8877f18e77506b2c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.052952 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbcf4b14b2274b7f8877f18e77506b2c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.053646 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbcf4b14b2274b7f8877f18e77506b2c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.066595 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1418/2850] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:39:24 2022] DELETE /identity/v3/users/bbcf4b14b2274b7f8877f18e77506b2c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:25.334100 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:25.335661 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:25.349578 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f2ae2-039a-422f-bb6b-89f01bae0c68 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:25.410423 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:25.411459 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:25.411750 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:25.412030 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] PATH_INFO: `/v3/projects/ba8972cf9a2841c9a46c1d2e70c036c9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:25.413308 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba8972cf9a2841c9a46c1d2e70c036c9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:25.415526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:25.464877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba8972cf9a2841c9a46c1d2e70c036c9', 'request_id': 'req-e3a5b651-249f-44dc-8b72-9503991fe723'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.474012 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba8972cf9a2841c9a46c1d2e70c036c9', 'request_id': 'req-e3a5b651-249f-44dc-8b72-9503991fe723'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.481435 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba8972cf9a2841c9a46c1d2e70c036c9', 'request_id': 'req-e3a5b651-249f-44dc-8b72-9503991fe723'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.492906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] The token cache is being invalidate because project ba8972cf9a2841c9a46c1d2e70c036c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:39:25.493209 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.494219 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.495920 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1433/2851] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:39:25 2022] DELETE /identity/v3/projects/ba8972cf9a2841c9a46c1d2e70c036c9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:25.500154 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:25.502476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:25.515804 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fc4ff-5a50-4ad7-8b3a-1820806f4594 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:25.575631 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e347e6-8f1d-44c8-8e03-d6ba3ee3d654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:25.576520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e347e6-8f1d-44c8-8e03-d6ba3ee3d654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:25.576933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e347e6-8f1d-44c8-8e03-d6ba3ee3d654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:25.577219 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e347e6-8f1d-44c8-8e03-d6ba3ee3d654 admin admin] PATH_INFO: `/v3/users/c5430580282244dd8198c0a4a49bf878` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:25.578476 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e347e6-8f1d-44c8-8e03-d6ba3ee3d654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5430580282244dd8198c0a4a49bf878)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:25.580573 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e347e6-8f1d-44c8-8e03-d6ba3ee3d654 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:25.623353 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:25.630737 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f2e347e6-8f1d-44c8-8e03-d6ba3ee3d654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5430580282244dd8198c0a4a49bf878'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.631713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f2e347e6-8f1d-44c8-8e03-d6ba3ee3d654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5430580282244dd8198c0a4a49bf878'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.640179 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f2e347e6-8f1d-44c8-8e03-d6ba3ee3d654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5430580282244dd8198c0a4a49bf878'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.653406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f2e347e6-8f1d-44c8-8e03-d6ba3ee3d654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5430580282244dd8198c0a4a49bf878'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.654149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f2e347e6-8f1d-44c8-8e03-d6ba3ee3d654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5430580282244dd8198c0a4a49bf878'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:25.669456 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1419/2852] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:39:25 2022] DELETE /identity/v3/users/c5430580282244dd8198c0a4a49bf878 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:26.028985 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:26.030724 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:26.043991 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5b651-249f-44dc-8b72-9503991fe723 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:26.106894 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:26.107761 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:26.108045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:26.108375 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] PATH_INFO: `/v3/projects/36bdd8c845da4b4d8ea9bbce2b877eca` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:26.109548 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36bdd8c845da4b4d8ea9bbce2b877eca)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:26.112073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:26.148402 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:26.166009 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36bdd8c845da4b4d8ea9bbce2b877eca', 'request_id': 'req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:26.173905 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36bdd8c845da4b4d8ea9bbce2b877eca', 'request_id': 'req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:26.180191 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36bdd8c845da4b4d8ea9bbce2b877eca', 'request_id': 'req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:26.191452 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] The token cache is being invalidate because project 36bdd8c845da4b4d8ea9bbce2b877eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:39:26.191950 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:26.192589 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c8ec0dcd-7356-4b3c-a6a0-8fba456ed3d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:39:26.194165 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1434/2853] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:39:26 2022] DELETE /identity/v3/projects/36bdd8c845da4b4d8ea9bbce2b877eca => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:26.199251 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a678055-a09a-4902-9981-9a48eb186811 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:26.199549 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a678055-a09a-4902-9981-9a48eb186811 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:26.199847 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a678055-a09a-4902-9981-9a48eb186811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:26.221790 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9a678055-a09a-4902-9981-9a48eb186811 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:26.223121 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9a678055-a09a-4902-9981-9a48eb186811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:26.284217 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1420/2854] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:26.291906 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0064-9216-4530-995b-0535388848f5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:26.292196 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0064-9216-4530-995b-0535388848f5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:26.292479 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0064-9216-4530-995b-0535388848f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:26.311226 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ff1b0064-9216-4530-995b-0535388848f5 None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:26.312616 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ff1b0064-9216-4530-995b-0535388848f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:26.321400 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1435/2855] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:26.325640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a678055-a09a-4902-9981-9a48eb186811 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:26.341772 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a678055-a09a-4902-9981-9a48eb186811 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:26.346957 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78425d-a1a8-446d-8431-581a127c7c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:26.347621 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78425d-a1a8-446d-8431-581a127c7c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:26.347906 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78425d-a1a8-446d-8431-581a127c7c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:26.348184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78425d-a1a8-446d-8431-581a127c7c2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:26.348914 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78425d-a1a8-446d-8431-581a127c7c2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:39:26.349225 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78425d-a1a8-446d-8431-581a127c7c2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:26.350867 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78425d-a1a8-446d-8431-581a127c7c2c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:26.363272 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1421/2856] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 06:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:39:26.367541 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b0064-9216-4530-995b-0535388848f5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:26.376927 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b0064-9216-4530-995b-0535388848f5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:26.381508 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25cf82-7f7c-415e-a4db-7bd98e1a61bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:26.382212 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25cf82-7f7c-415e-a4db-7bd98e1a61bc admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:26.382499 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25cf82-7f7c-415e-a4db-7bd98e1a61bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:26.382778 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25cf82-7f7c-415e-a4db-7bd98e1a61bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:26.383480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d25cf82-7f7c-415e-a4db-7bd98e1a61bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:26.385383 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d25cf82-7f7c-415e-a4db-7bd98e1a61bc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:26.411322 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1436/2857] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 06:39:26 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:39:26.415335 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78425d-a1a8-446d-8431-581a127c7c2c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:26.425416 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78425d-a1a8-446d-8431-581a127c7c2c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:26.481821 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caa8c7-13af-444d-a337-16631d2fdf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:26.482518 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caa8c7-13af-444d-a337-16631d2fdf71 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:26.482800 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caa8c7-13af-444d-a337-16631d2fdf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:26.483075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caa8c7-13af-444d-a337-16631d2fdf71 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:26.483764 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5caa8c7-13af-444d-a337-16631d2fdf71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:26.486245 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5caa8c7-13af-444d-a337-16631d2fdf71 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:26.506182 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1422/2858] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 06:39:26 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:39:26.510245 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25cf82-7f7c-415e-a4db-7bd98e1a61bc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:26.520936 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25cf82-7f7c-415e-a4db-7bd98e1a61bc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:26.525820 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04b617-2627-4cb9-82fa-b7ec112a3c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:26.526523 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04b617-2627-4cb9-82fa-b7ec112a3c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:26.526806 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04b617-2627-4cb9-82fa-b7ec112a3c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:26.527084 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04b617-2627-4cb9-82fa-b7ec112a3c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:26.527812 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04b617-2627-4cb9-82fa-b7ec112a3c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:26.529755 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04b617-2627-4cb9-82fa-b7ec112a3c68 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:26.540680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1437/2859] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:39:26.544521 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caa8c7-13af-444d-a337-16631d2fdf71 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:26.554491 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caa8c7-13af-444d-a337-16631d2fdf71 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:26.559147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e82240-340a-4ba2-9726-1e3829bd2e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:26.559807 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e82240-340a-4ba2-9726-1e3829bd2e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:26.560095 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e82240-340a-4ba2-9726-1e3829bd2e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:26.560378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e82240-340a-4ba2-9726-1e3829bd2e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:26.561112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e82240-340a-4ba2-9726-1e3829bd2e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:26.562781 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e82240-340a-4ba2-9726-1e3829bd2e2f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:26.572688 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1423/2860] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 06:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:39:26.576838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04b617-2627-4cb9-82fa-b7ec112a3c68 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:26.587902 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04b617-2627-4cb9-82fa-b7ec112a3c68 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:26.592620 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8fadb-83ad-46e7-8afa-75b7de51a651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:26.593492 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8fadb-83ad-46e7-8afa-75b7de51a651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:26.593775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8fadb-83ad-46e7-8afa-75b7de51a651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:26.594090 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8fadb-83ad-46e7-8afa-75b7de51a651 admin admin] PATH_INFO: `/v3/projects/632f7d981de1403997f65df54b79310e/users/cbb7ad921d564f30b9e310f2890ea96c/roles/d3c75d38368d47d799bf59a0f9c879ea` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:26.609126 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8fadb-83ad-46e7-8afa-75b7de51a651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=632f7d981de1403997f65df54b79310e, user_id=cbb7ad921d564f30b9e310f2890ea96c, role_id=d3c75d38368d47d799bf59a0f9c879ea)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:26.612021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8fadb-83ad-46e7-8afa-75b7de51a651 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:26.624611 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1438/2861] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 06:39:26 2022] PUT /identity/v3/projects/632f7d981de1403997f65df54b79310e/users/cbb7ad921d564f30b9e310f2890ea96c/roles/d3c75d38368d47d799bf59a0f9c879ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:39:26.635075 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37903fa1-03a5-477d-ac56-dd6badbbb25e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:26.635367 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37903fa1-03a5-477d-ac56-dd6badbbb25e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:26.635648 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37903fa1-03a5-477d-ac56-dd6badbbb25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:26.787634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-37903fa1-03a5-477d-ac56-dd6badbbb25e None None] MFA Rules not processed for user `cbb7ad921d564f30b9e310f2890ea96c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:39:26.788849 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-37903fa1-03a5-477d-ac56-dd6badbbb25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:26.798024 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:26.853771 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1424/2862] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:39:26 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:39:26.864797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8fadb-83ad-46e7-8afa-75b7de51a651 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:26.866497 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-54b8fadb-83ad-46e7-8afa-75b7de51a651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:26.924141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8fadb-83ad-46e7-8afa-75b7de51a651 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:26.928315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda50b8-2e20-4319-8ace-45892ef4f85e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '710c422b8024403da666ae5054ba1289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:26.928967 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda50b8-2e20-4319-8ace-45892ef4f85e service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:26.929250 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda50b8-2e20-4319-8ace-45892ef4f85e service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:26.929532 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda50b8-2e20-4319-8ace-45892ef4f85e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:26.938918 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda50b8-2e20-4319-8ace-45892ef4f85e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:26.940409 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda50b8-2e20-4319-8ace-45892ef4f85e service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:26.949538 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1439/2863] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:36.850885 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=22 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:36.945253 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:39:38.324784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37903fa1-03a5-477d-ac56-dd6badbbb25e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:39:38.326503 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-37903fa1-03a5-477d-ac56-dd6badbbb25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:38.412062 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37903fa1-03a5-477d-ac56-dd6badbbb25e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:39:38.414544 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e575fa-92fe-46d8-98b9-7d59c59ff9f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:39:38.415230 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e575fa-92fe-46d8-98b9-7d59c59ff9f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:39:38.415481 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e575fa-92fe-46d8-98b9-7d59c59ff9f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:39:38.415837 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e575fa-92fe-46d8-98b9-7d59c59ff9f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:39:38.418303 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-12e575fa-92fe-46d8-98b9-7d59c59ff9f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:39:38.484374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e575fa-92fe-46d8-98b9-7d59c59ff9f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:39:38.485854 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e575fa-92fe-46d8-98b9-7d59c59ff9f4 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:39:38.496205 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1425/2864] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:39:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:39:48.491267 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=50 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:40:02.382378 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda50b8-2e20-4319-8ace-45892ef4f85e service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:02.383988 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cdda50b8-2e20-4319-8ace-45892ef4f85e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:02.396309 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda50b8-2e20-4319-8ace-45892ef4f85e service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:02.399939 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d520d05-22ed-47f6-b7cf-ae722bf01e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:02.400581 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520d05-22ed-47f6-b7cf-ae722bf01e3e service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:02.400872 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520d05-22ed-47f6-b7cf-ae722bf01e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:02.401173 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520d05-22ed-47f6-b7cf-ae722bf01e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:02.403001 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1d520d05-22ed-47f6-b7cf-ae722bf01e3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:02.413970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d520d05-22ed-47f6-b7cf-ae722bf01e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:02.415425 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d520d05-22ed-47f6-b7cf-ae722bf01e3e service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:02.424783 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1440/2865] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:40:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:40:02.527886 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e575fa-92fe-46d8-98b9-7d59c59ff9f4 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:02.538690 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e575fa-92fe-46d8-98b9-7d59c59ff9f4 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:02.542520 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4510ce-44e0-4b03-9636-789cc53cf601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:02.543415 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4510ce-44e0-4b03-9636-789cc53cf601 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:02.543860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4510ce-44e0-4b03-9636-789cc53cf601 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:02.544296 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4510ce-44e0-4b03-9636-789cc53cf601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:02.546356 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dc4510ce-44e0-4b03-9636-789cc53cf601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:02.625163 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4510ce-44e0-4b03-9636-789cc53cf601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:02.626634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4510ce-44e0-4b03-9636-789cc53cf601 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:02.637917 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1426/2866] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:40:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:40:04.132138 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d520d05-22ed-47f6-b7cf-ae722bf01e3e service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:04.196692 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d520d05-22ed-47f6-b7cf-ae722bf01e3e service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:04.201223 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:04.203019 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:04.203486 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:04.203775 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] PATH_INFO: `/v3/users/cbb7ad921d564f30b9e310f2890ea96c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:04.205024 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb7ad921d564f30b9e310f2890ea96c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:04.207008 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:04.253673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb7ad921d564f30b9e310f2890ea96c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:04.266417 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb7ad921d564f30b9e310f2890ea96c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:04.273169 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb7ad921d564f30b9e310f2890ea96c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:04.273877 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb7ad921d564f30b9e310f2890ea96c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:04.275043 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb7ad921d564f30b9e310f2890ea96c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:04.288613 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1441/2867] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:40:04 2022] DELETE /identity/v3/users/cbb7ad921d564f30b9e310f2890ea96c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:40:04.293263 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4480536-d2d7-4a2f-acea-60dbe4a6cbae None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:04.293779 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4480536-d2d7-4a2f-acea-60dbe4a6cbae None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:04.293934 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4480536-d2d7-4a2f-acea-60dbe4a6cbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:04.312728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f4480536-d2d7-4a2f-acea-60dbe4a6cbae None None] MFA Rules not processed for user `a1033721c1da485882816b21d7d01a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:40:04.313979 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f4480536-d2d7-4a2f-acea-60dbe4a6cbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:04.377810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1427/2868] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 06:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:40:04.387567 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:04.388976 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:04.448965 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6245f3-a1e8-4c12-b1bd-9a298e3a49d5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:04.453454 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ed89e-4010-4f3f-856b-018762f8599d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:04.453949 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ed89e-4010-4f3f-856b-018762f8599d service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:04.454283 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ed89e-4010-4f3f-856b-018762f8599d service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:04.454586 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ed89e-4010-4f3f-856b-018762f8599d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:04.464583 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ed89e-4010-4f3f-856b-018762f8599d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:04.466061 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ed89e-4010-4f3f-856b-018762f8599d service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:04.476256 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1442/2869] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:40:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:40:04.756554 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4480536-d2d7-4a2f-acea-60dbe4a6cbae None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:04.758112 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f4480536-d2d7-4a2f-acea-60dbe4a6cbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:04.776480 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4480536-d2d7-4a2f-acea-60dbe4a6cbae None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:04.784382 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:04.785277 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:04.785581 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:04.786018 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] PATH_INFO: `/v3/projects/632f7d981de1403997f65df54b79310e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:04.787233 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=632f7d981de1403997f65df54b79310e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:04.789220 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:04.841728 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '632f7d981de1403997f65df54b79310e', 'request_id': 'req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:04.848391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '632f7d981de1403997f65df54b79310e', 'request_id': 'req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:04.855732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '632f7d981de1403997f65df54b79310e', 'request_id': 'req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:04.872750 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] The token cache is being invalidate because project 632f7d981de1403997f65df54b79310e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:40:04.873130 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:04.873810 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:04.875641 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1428/2870] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:40:04 2022] DELETE /identity/v3/projects/632f7d981de1403997f65df54b79310e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:40:12.524124 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ed89e-4010-4f3f-856b-018762f8599d service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:12.534763 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ed89e-4010-4f3f-856b-018762f8599d service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:12.540179 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:40:12.600415 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f9513-e062-4893-96e5-377df51c0305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:12.601392 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f9513-e062-4893-96e5-377df51c0305 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:12.601680 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f9513-e062-4893-96e5-377df51c0305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:12.601997 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f9513-e062-4893-96e5-377df51c0305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:12.603800 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cb9f9513-e062-4893-96e5-377df51c0305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:12.677743 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f9513-e062-4893-96e5-377df51c0305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:12.679260 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f9513-e062-4893-96e5-377df51c0305 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:12.690797 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1443/2871] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:40:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:40:14.869705 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:40:18.626506 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:18.628014 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:18.640051 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3eb5f-aad8-4f98-aebd-3ef1e1687a9a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:18.644464 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6812974-99e7-4a14-a3fd-01b208946461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:18.645100 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6812974-99e7-4a14-a3fd-01b208946461 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:18.645386 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6812974-99e7-4a14-a3fd-01b208946461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:18.645666 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6812974-99e7-4a14-a3fd-01b208946461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:18.647423 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b6812974-99e7-4a14-a3fd-01b208946461 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:18.660963 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6812974-99e7-4a14-a3fd-01b208946461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:18.662427 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6812974-99e7-4a14-a3fd-01b208946461 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:18.673933 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1429/2872] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:40:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:40:18.942235 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f9513-e062-4893-96e5-377df51c0305 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:18.951045 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f9513-e062-4893-96e5-377df51c0305 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:18.955134 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd404e61-6a74-4eac-a9f6-4599a30bef48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:18.955779 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd404e61-6a74-4eac-a9f6-4599a30bef48 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:18.956063 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd404e61-6a74-4eac-a9f6-4599a30bef48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:18.956342 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd404e61-6a74-4eac-a9f6-4599a30bef48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:18.958319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fd404e61-6a74-4eac-a9f6-4599a30bef48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:19.021953 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd404e61-6a74-4eac-a9f6-4599a30bef48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:19.023407 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd404e61-6a74-4eac-a9f6-4599a30bef48 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:19.033519 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1444/2873] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:40:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:40:25.488030 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6812974-99e7-4a14-a3fd-01b208946461 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:25.489504 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b6812974-99e7-4a14-a3fd-01b208946461 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:25.507082 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6812974-99e7-4a14-a3fd-01b208946461 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:25.512131 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:25.513032 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:25.513321 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:25.513605 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] PATH_INFO: `/v3/users/5439d1a65154486bb4fdfeeaf8ab4b07` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:25.514915 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5439d1a65154486bb4fdfeeaf8ab4b07)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:25.517197 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:25.571713 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5439d1a65154486bb4fdfeeaf8ab4b07'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:25.572463 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5439d1a65154486bb4fdfeeaf8ab4b07'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:25.580611 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5439d1a65154486bb4fdfeeaf8ab4b07'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:25.592432 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5439d1a65154486bb4fdfeeaf8ab4b07'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:25.593139 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5439d1a65154486bb4fdfeeaf8ab4b07'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:25.606715 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1430/2874] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:40:25 2022] DELETE /identity/v3/users/5439d1a65154486bb4fdfeeaf8ab4b07 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:40:25.783700 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd404e61-6a74-4eac-a9f6-4599a30bef48 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:25.804631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd404e61-6a74-4eac-a9f6-4599a30bef48 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:25.864598 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:40:25.879543 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:25.880450 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:25.880759 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:25.881032 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] PATH_INFO: `/v3/projects/5d0ba27e9a7f450c87eb189cdd6e235b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:25.882339 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d0ba27e9a7f450c87eb189cdd6e235b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:25.884270 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:25.941737 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d0ba27e9a7f450c87eb189cdd6e235b', 'request_id': 'req-23a026a5-34f7-4ba5-ae95-582ab857b616'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:25.948923 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d0ba27e9a7f450c87eb189cdd6e235b', 'request_id': 'req-23a026a5-34f7-4ba5-ae95-582ab857b616'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:25.955891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d0ba27e9a7f450c87eb189cdd6e235b', 'request_id': 'req-23a026a5-34f7-4ba5-ae95-582ab857b616'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:25.966310 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] The token cache is being invalidate because project 5d0ba27e9a7f450c87eb189cdd6e235b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:40:25.966600 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:25.967241 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:25.969890 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1445/2875] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:40:25 2022] DELETE /identity/v3/projects/5d0ba27e9a7f450c87eb189cdd6e235b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:40:25.973602 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:25.975089 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:25.988494 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efce01-6cd8-42fe-8226-9cb7c6079895 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:26.045435 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccd9ee-742c-4cae-aaa7-6896a21ed5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:26.046038 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccd9ee-742c-4cae-aaa7-6896a21ed5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:26.046322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccd9ee-742c-4cae-aaa7-6896a21ed5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:26.046597 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccd9ee-742c-4cae-aaa7-6896a21ed5bf admin admin] PATH_INFO: `/v3/users/9738703dbbc74b758733f2ea6858b886` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:26.047812 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ccd9ee-742c-4cae-aaa7-6896a21ed5bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9738703dbbc74b758733f2ea6858b886)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:26.049708 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ccd9ee-742c-4cae-aaa7-6896a21ed5bf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:26.096634 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d7ccd9ee-742c-4cae-aaa7-6896a21ed5bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9738703dbbc74b758733f2ea6858b886'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:26.097283 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d7ccd9ee-742c-4cae-aaa7-6896a21ed5bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9738703dbbc74b758733f2ea6858b886'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:26.104074 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d7ccd9ee-742c-4cae-aaa7-6896a21ed5bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9738703dbbc74b758733f2ea6858b886'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:26.108433 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:40:26.118378 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d7ccd9ee-742c-4cae-aaa7-6896a21ed5bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9738703dbbc74b758733f2ea6858b886'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:26.119007 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d7ccd9ee-742c-4cae-aaa7-6896a21ed5bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9738703dbbc74b758733f2ea6858b886'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:26.131652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1431/2876] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:40:25 2022] DELETE /identity/v3/users/9738703dbbc74b758733f2ea6858b886 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:40:26.318239 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:26.319699 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:26.334919 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a026a5-34f7-4ba5-ae95-582ab857b616 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:26.394350 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:26.395221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:26.395507 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:26.395789 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] PATH_INFO: `/v3/projects/94470f28d9c0453bae57cb45691072ba` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:26.396982 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=94470f28d9c0453bae57cb45691072ba)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:26.398945 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:26.445346 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94470f28d9c0453bae57cb45691072ba', 'request_id': 'req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:26.453274 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94470f28d9c0453bae57cb45691072ba', 'request_id': 'req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:26.459394 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94470f28d9c0453bae57cb45691072ba', 'request_id': 'req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:26.469998 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] The token cache is being invalidate because project 94470f28d9c0453bae57cb45691072ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:40:26.470288 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:26.470924 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:40:26.472457 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1446/2877] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:40:26 2022] DELETE /identity/v3/projects/94470f28d9c0453bae57cb45691072ba => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:40:36.128106 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:40:36.466526 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:40:59.710172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51186a4f-ac52-4e2b-a0e5-3e5b91f69eeb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:59.710680 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51186a4f-ac52-4e2b-a0e5-3e5b91f69eeb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:59.710784 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51186a4f-ac52-4e2b-a0e5-3e5b91f69eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:59.758594 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-51186a4f-ac52-4e2b-a0e5-3e5b91f69eeb None None] MFA Rules not processed for user `e0ff633636ec403780dd4bd21e0fa65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:40:59.759794 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-51186a4f-ac52-4e2b-a0e5-3e5b91f69eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:59.824406 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1432/2878] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jul 21 06:40:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:40:59.864298 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:40:59.865703 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:40:59.962891 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71bac4-1d74-4951-a9cb-d1c61cec4e7e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:40:59.967984 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c293d-b1c0-46e2-908e-b6b09ecbbd99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94c8eb61f2ca48288cba81d280a30747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:40:59.968631 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c293d-b1c0-46e2-908e-b6b09ecbbd99 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:40:59.968912 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c293d-b1c0-46e2-908e-b6b09ecbbd99 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:40:59.969189 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c293d-b1c0-46e2-908e-b6b09ecbbd99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:40:59.978221 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c293d-b1c0-46e2-908e-b6b09ecbbd99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:40:59.979785 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c293d-b1c0-46e2-908e-b6b09ecbbd99 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:40:59.987758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1447/2879] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 06:40:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:41:06.344593 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51186a4f-ac52-4e2b-a0e5-3e5b91f69eeb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:06.346349 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-51186a4f-ac52-4e2b-a0e5-3e5b91f69eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:06.420722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51186a4f-ac52-4e2b-a0e5-3e5b91f69eeb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:06.425184 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:06.426101 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:06.426387 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:06.426668 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] PATH_INFO: `/v3/users/756c74602c4d4bafade01c406ffd6afc` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:06.428659 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=756c74602c4d4bafade01c406ffd6afc)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:06.430641 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:06.476170 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '756c74602c4d4bafade01c406ffd6afc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.476819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '756c74602c4d4bafade01c406ffd6afc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.483552 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '756c74602c4d4bafade01c406ffd6afc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.495571 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '756c74602c4d4bafade01c406ffd6afc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.496203 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '756c74602c4d4bafade01c406ffd6afc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.497855 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:41:06.510077 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1433/2880] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:41:06 2022] DELETE /identity/v3/users/756c74602c4d4bafade01c406ffd6afc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:41:06.681234 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c293d-b1c0-46e2-908e-b6b09ecbbd99 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:06.692021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c293d-b1c0-46e2-908e-b6b09ecbbd99 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:06.750195 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:06.751040 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:06.751319 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:06.751594 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] PATH_INFO: `/v3/projects/9ba482e050244bc6a359090a616658ec` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:06.752758 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ba482e050244bc6a359090a616658ec)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:06.754697 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:06.801055 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ba482e050244bc6a359090a616658ec', 'request_id': 'req-cff7853a-062e-4b92-8202-a1cbb7f15c73'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.809021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ba482e050244bc6a359090a616658ec', 'request_id': 'req-cff7853a-062e-4b92-8202-a1cbb7f15c73'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.815073 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ba482e050244bc6a359090a616658ec', 'request_id': 'req-cff7853a-062e-4b92-8202-a1cbb7f15c73'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.823026 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:41:06.827315 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] The token cache is being invalidate because project 9ba482e050244bc6a359090a616658ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:41:06.827718 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.828656 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.830673 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1448/2881] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:41:06 2022] DELETE /identity/v3/projects/9ba482e050244bc6a359090a616658ec => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:41:06.835105 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:06.836538 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:06.850908 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e5ea9-0397-435d-8e03-62dd6f2aaa07 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:06.906141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:06.906929 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:06.907210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:06.907484 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] PATH_INFO: `/v3/users/941e6b2a346d4a93ad614de89460d7b2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:06.908926 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=941e6b2a346d4a93ad614de89460d7b2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:06.910858 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:06.957149 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '941e6b2a346d4a93ad614de89460d7b2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.957803 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '941e6b2a346d4a93ad614de89460d7b2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.965447 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '941e6b2a346d4a93ad614de89460d7b2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.977652 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '941e6b2a346d4a93ad614de89460d7b2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.978318 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '941e6b2a346d4a93ad614de89460d7b2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:06.991172 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1434/2882] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:41:06 2022] DELETE /identity/v3/users/941e6b2a346d4a93ad614de89460d7b2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:41:07.166232 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:07.167672 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:07.181004 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7853a-062e-4b92-8202-a1cbb7f15c73 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:07.241128 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:07.242034 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:07.242325 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:07.242607 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] PATH_INFO: `/v3/projects/f58c7912c6b24db6b34dde825f9e6c04` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:07.243801 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f58c7912c6b24db6b34dde825f9e6c04)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:07.245716 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:07.292638 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f58c7912c6b24db6b34dde825f9e6c04', 'request_id': 'req-ba4b5022-b44d-4250-8dd9-b88f47f87f04'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:07.299515 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f58c7912c6b24db6b34dde825f9e6c04', 'request_id': 'req-ba4b5022-b44d-4250-8dd9-b88f47f87f04'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:07.306511 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f58c7912c6b24db6b34dde825f9e6c04', 'request_id': 'req-ba4b5022-b44d-4250-8dd9-b88f47f87f04'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:07.316735 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] The token cache is being invalidate because project f58c7912c6b24db6b34dde825f9e6c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:41:07.317021 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:07.318281 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:07.319957 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1449/2883] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:41:07 2022] DELETE /identity/v3/projects/f58c7912c6b24db6b34dde825f9e6c04 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:41:16.987374 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=55 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:41:17.313819 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:41:36.172301 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:36.174147 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:36.261210 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b59d91-d302-47c7-a52b-c615a9b89ba3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:36.342340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc5a25-4df1-442a-991f-ee659d1381e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f099e6f7545a1b2734dbe5afd2027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:36.342630 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc5a25-4df1-442a-991f-ee659d1381e1 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:36.342910 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc5a25-4df1-442a-991f-ee659d1381e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:36.343239 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc5a25-4df1-442a-991f-ee659d1381e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:36.345063 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8fcc5a25-4df1-442a-991f-ee659d1381e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:36.358206 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc5a25-4df1-442a-991f-ee659d1381e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:36.359839 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc5a25-4df1-442a-991f-ee659d1381e1 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:36.372166 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1435/2884] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:41:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:41:37.416259 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:37.417648 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:37.474838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5022-b44d-4250-8dd9-b88f47f87f04 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:37.478482 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25e7b0-db47-4402-abaf-c20ed5894af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:37.479141 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25e7b0-db47-4402-abaf-c20ed5894af8 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:37.479427 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25e7b0-db47-4402-abaf-c20ed5894af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:37.479707 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25e7b0-db47-4402-abaf-c20ed5894af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:37.481480 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ed25e7b0-db47-4402-abaf-c20ed5894af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:37.491981 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25e7b0-db47-4402-abaf-c20ed5894af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:37.493416 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25e7b0-db47-4402-abaf-c20ed5894af8 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:37.501728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1450/2885] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:41:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:41:45.688337 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc5a25-4df1-442a-991f-ee659d1381e1 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:45.690029 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8fcc5a25-4df1-442a-991f-ee659d1381e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:45.703547 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc5a25-4df1-442a-991f-ee659d1381e1 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:45.707333 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29642d01-6e58-4fbf-8ca4-63fe24a4fc2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a9c3f34eaeb4b538528ff9050252dc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:45.707860 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29642d01-6e58-4fbf-8ca4-63fe24a4fc2e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:45.708141 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29642d01-6e58-4fbf-8ca4-63fe24a4fc2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:45.708417 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29642d01-6e58-4fbf-8ca4-63fe24a4fc2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:45.710838 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-29642d01-6e58-4fbf-8ca4-63fe24a4fc2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:45.780729 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29642d01-6e58-4fbf-8ca4-63fe24a4fc2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:45.782955 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29642d01-6e58-4fbf-8ca4-63fe24a4fc2e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:45.795449 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1436/2886] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:41:46.581464 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25e7b0-db47-4402-abaf-c20ed5894af8 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:46.583585 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ed25e7b0-db47-4402-abaf-c20ed5894af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:46.657701 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29642d01-6e58-4fbf-8ca4-63fe24a4fc2e service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:46.659371 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-29642d01-6e58-4fbf-8ca4-63fe24a4fc2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:46.672604 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25e7b0-db47-4402-abaf-c20ed5894af8 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:46.675284 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29642d01-6e58-4fbf-8ca4-63fe24a4fc2e service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:46.677807 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c928d80-73c5-4086-9b99-bea218565b07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da2d85eb18049f196aeefc51d3f1e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:46.678491 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c928d80-73c5-4086-9b99-bea218565b07 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:46.678818 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c928d80-73c5-4086-9b99-bea218565b07 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:46.679722 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c928d80-73c5-4086-9b99-bea218565b07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:46.680569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:46.681546 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:46.681675 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2c928d80-73c5-4086-9b99-bea218565b07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:46.681835 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:46.682264 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] PATH_INFO: `/v3/users/0f951e37e60145aba97bcd1ff8aad260` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:46.683467 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f951e37e60145aba97bcd1ff8aad260)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:46.685391 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:46.693268 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c928d80-73c5-4086-9b99-bea218565b07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:46.696007 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c928d80-73c5-4086-9b99-bea218565b07 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:46.700265 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:41:46.707704 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1451/2887] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:41:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:41:46.736569 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f951e37e60145aba97bcd1ff8aad260'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:46.737221 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f951e37e60145aba97bcd1ff8aad260'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:46.747261 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f951e37e60145aba97bcd1ff8aad260'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:46.759330 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f951e37e60145aba97bcd1ff8aad260'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:46.760047 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f951e37e60145aba97bcd1ff8aad260'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:46.773226 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1437/2888] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:41:46 2022] DELETE /identity/v3/users/0f951e37e60145aba97bcd1ff8aad260 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:41:46.955203 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c928d80-73c5-4086-9b99-bea218565b07 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:46.968830 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c928d80-73c5-4086-9b99-bea218565b07 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:47.041213 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:47.042125 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:47.042447 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:47.042728 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] PATH_INFO: `/v3/projects/fd3a6e00103d49038260d28a680a9c56` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:47.043968 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3a6e00103d49038260d28a680a9c56)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:47.045992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:47.082698 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:41:47.093970 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3a6e00103d49038260d28a680a9c56', 'request_id': 'req-a4febe34-fc39-4d32-9fa9-ffb19da3988d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.101930 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3a6e00103d49038260d28a680a9c56', 'request_id': 'req-a4febe34-fc39-4d32-9fa9-ffb19da3988d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.108608 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3a6e00103d49038260d28a680a9c56', 'request_id': 'req-a4febe34-fc39-4d32-9fa9-ffb19da3988d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.118838 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] The token cache is being invalidate because project fd3a6e00103d49038260d28a680a9c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:41:47.119167 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.119825 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.121366 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1452/2889] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:41:46 2022] DELETE /identity/v3/projects/fd3a6e00103d49038260d28a680a9c56 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:41:47.125322 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:47.126724 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:47.139722 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c74550-adf4-4037-abe0-8fb187103404 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:47.197278 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:47.198235 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:47.198459 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:47.198732 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] PATH_INFO: `/v3/users/99c87dbc5a9346ff80537975a92fa4f4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:47.199918 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99c87dbc5a9346ff80537975a92fa4f4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:47.201819 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:47.249115 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99c87dbc5a9346ff80537975a92fa4f4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.249773 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99c87dbc5a9346ff80537975a92fa4f4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.256617 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99c87dbc5a9346ff80537975a92fa4f4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.269253 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99c87dbc5a9346ff80537975a92fa4f4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.269920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99c87dbc5a9346ff80537975a92fa4f4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.282143 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1438/2890] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 06:41:47 2022] DELETE /identity/v3/users/99c87dbc5a9346ff80537975a92fa4f4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:41:47.453280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:41:47.455223 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:41:47.468439 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4febe34-fc39-4d32-9fa9-ffb19da3988d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:41:47.527243 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1033721c1da485882816b21d7d01a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c098b85eefc14c3cac1c96ebb357b960', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:41:47.528137 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:41:47.528438 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:41:47.528723 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] PATH_INFO: `/v3/projects/f188a0b484cc4781953e86161e579322` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:41:47.529912 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f188a0b484cc4781953e86161e579322)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:41:47.531848 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:41:47.579391 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f188a0b484cc4781953e86161e579322', 'request_id': 'req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.586380 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f188a0b484cc4781953e86161e579322', 'request_id': 'req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.592679 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f188a0b484cc4781953e86161e579322', 'request_id': 'req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.603992 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] The token cache is being invalidate because project f188a0b484cc4781953e86161e579322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:41:47.604280 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.604909 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6f8daa99-101f-4c9b-ab8a-1194a14daf1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:41:47.606470 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1453/2891] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 06:41:47 2022] DELETE /identity/v3/projects/f188a0b484cc4781953e86161e579322 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:41:57.278585 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=75,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:41:57.600110 dv-cinder-850318-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:44:28.141512 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:44:28.143024 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:44:28.212718 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc488e51-d992-40f6-b154-c0870e055e44 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:44:28.271696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb329d-c206-4c46-9dbe-e90a9cbf3a60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da2d85eb18049f196aeefc51d3f1e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91ccc1b22a434cf8a9be253de3ee3862', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:44:28.272340 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb329d-c206-4c46-9dbe-e90a9cbf3a60 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:44:28.272622 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb329d-c206-4c46-9dbe-e90a9cbf3a60 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:44:28.272898 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb329d-c206-4c46-9dbe-e90a9cbf3a60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:44:28.274696 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7adb329d-c206-4c46-9dbe-e90a9cbf3a60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:44:28.286358 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb329d-c206-4c46-9dbe-e90a9cbf3a60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:44:28.287920 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb329d-c206-4c46-9dbe-e90a9cbf3a60 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:44:28.297871 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1439/2892] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 06:44:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:44:38.293640 dv-cinder-850318-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=58 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}